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ABSTRACT 


Systems  engineering  is  vital  for  the  acquisition  of  systems  for  the  Department  of  the 
Navy  (DON).  As  systems  engineering  is  a  relatively  young  discipline,  no  professional 
engineer  occupational  series  exists  under  which  systems  engineers  can  be  classified  from 
a  human  resources  perspective.  In  addition  to  the  lack  of  an  occupational  designation, 
there  is  no  official  competency  model  to  form  the  basis  for  employee  selection  and  career 
development.  In  order  for  a  competency  model  to  be  used  for  employee  selection,  it  must 
be  validated  under  the  Uniform  Guidelines  for  employee  selection.  Once  validated,  the 
model  can  be  used  to  create  systems  engineering  position  descriptions  and  related  career 
development  plans  that  would  be  specifically  used  for  systems  engineers  within  the  DON 
and  perhaps  for  DOD.  A  baseline  competency  model  is  the  first  step  in  performing  a 
validation  process  in  accordance  with  the  Uniform  Guidelines. 

In  order  to  begin  to  address  this  situation,  a  system  engineering  competency 
model  was  developed  to  provide  a  baseline.  This  model  was  designed  specifically  for  the 
DON,  though  it  should  also  be  useable  in  any  organization  that  employs  system 
engineers.  The  core  of  the  model  is  based  on  the  knowledge,  skills,  and  abilities  (KSAs) 
that  a  systems  engineer  needs  to  develop  competency  in  across  varying  career  levels. 
These  KSAs  are  defined  using  Bloom’s  Taxonomy  to  describe,  the  cognitive  and 
affective  aspects  needed  for  achievement  of  the  respective  competencies.  The  research 
also  identifies  whether  these  KSAs  are  best  attained  through  specific  methods,  such  as 
undergraduate  education,  graduate  education,  professional  training,  or  through  on-the-job 
experience.  Furthermore,  the  model  can  inform  the  development  of  graduate  and 
undergraduate  curricula  in  systems  engineering,  since  using  Bloom’s  Taxonomy 
describes  the  KSAs  in  terms  that  lend  themselves  to  direct  use  as  curriculum  related 
learning  objectives. 

The  outcome  is  called  the  Competency  Model  for  the  Profession  of  Systems 
Engineering  (COMPOSE)  model.  The  COMPOSE  model  can  be  used  by  the  DON  and 
several  other  organizations  as  a  means  to  formulate  career  development  plans  for  the 
professional  development  of  systems  engineers. 
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EXECUTIVE  SUMMARY 


The  Competency  of  the  Profession  of  Systems  Engineering  (COMPOSE)  model  was 
designed  to  assist  with  career  development  modeling  and  creating  position  descriptions 
for  the  Department  of  Defense  (DOD)  (Whitcomb,  Khan,  &  White,  2013).  It  can  also 
assist  graduate  academic  programs  to  specify  objectives  within  systems  engineering  (SE) 
programs  that  will  ensure  the  students  have  the  entry-level  knowledge,  skills  and  abilities 
(KSAs)  required  to  perform  successfully  in  their  job.  Training  levels  and  competency 
sources  are  identified  within  the  model.  Each  KSA  was  mapped  and  analyzed  using  a 
Bloom’s  Taxonomy  cognitive  and  affective  domain  approach.  This  approach  provides  an 
interactive  model  that  determines  the  KSAs  required  for  DOD  systems  engineers  to  be 
considered  competent  at  various  career  experience  levels. 

The  COMPOSE  model  encompasses  eight  different  documented  systems 
engineering  competency  models  from  a  variety  of  organizations.  These  other 
competency  models  include  The  International  Council  on  Systems  Engineering 
(INCOSE)  United  Kingdom  (UK),  Boeing,  The  National  Aeronautics  and  Space 
Administration  (NASA),  Defense  Acquisition  University  (DAU)  Systems  Planning, 
Research,  Development  and  Engineering  (SPRDE),  Naval  Aviation  Systems  Command 
(NAVAIR),  MITRE,  Boeing,  Space  and  Naval  Warfare  Systems  Command  (SPAWAR) 
and  the  Naval  Underwater  Warfare  Center  (NUWC)  Newport.  The  COMPOSE  model 
uses  these  eight  models  as  a  foundation.  The  KSAs  are  harmonized  with  Bloom’s 
Taxonomy  based  on  affinity  in  the  KSAs.  The  KSAs  are  also  realigned  to  maintain 
consistency  in  the  COMPOSE  model  by  eliminating  duplication;  items  that  do  not  fit  are 
re-categorized  based  on  how  each  KSAs  is  written.  In  the  beginning,  after  initially 
combining  each  of  the  models,  there  were  2,151  KSAs  and  31  competencies.  Based  on 
the  realignment  and  re-categorization,  the  COMPOSE  0.78  model  has  a  total  of  41 
competencies  and  2,914  KSAs. 

The  KSAs  within  the  COMPOSE  model  are  also  characterized  to  align  with  either 
the  technical/  technical  management  or  the  professional  competencies.  Analysis  results 
indicate  that  when  it  comes  to  technical/technical  management  competency  within 


xv 


systems  engineering,  at  entry-level  positions  (SE-01)  lower  level  KSAs  from  the 
cognitive  domain  are  required.  As  the  career  level  increases,  so  does  the  complexity  of 
the  KSAs  within  the  cognitive  domain.  The  opposite  is  true  for  professional  competence 
within  the  SE  domain;  at  entry-level  positions,  KSAs  within  the  affective  domain  are 
needed  to  be  competent.  The  majority  of  the  KSAs  are  knowledge  and  comprehension 
based  within  the  cognitive  domain,  which  makes  sense  since  these  are  lower  level 
cognitive  domains  that  can  be  learned  by  training  and  education.  As  the  career  progresses 
in  journey-level  (SE-02)  and  expert  level  (SE-03)  career  levels,  the  focus  shifts  to 
application.  At  this  stage  in  the  career  development,  the  individual  is  required  to  apply 
what  was  learned  to  do  his/her  job.  This  means  that  all  expert  level  SE  position 
descriptions  should  substantially  highlight  application. 
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I. 


INTRODUCTION 


This  chapter  will  define  competency  and  explain  why  it  is  relevant  to  study  for 
systems  engineering  (SE).  It  will  also  share  the  attributes  found  in  a  good  competency 
model,  while  identifying  how  the  Competency  Model  for  the  Profession  of  Systems 
Engineering  (COMPOSE)  has  evolved. 

The  Systems  Planning,  Research,  Development  and  Engineering  (SPRDE)  career 
field  has  approximately  38,000  employees  (Lasley-Hunter  and  Alan  2011).  There  is 
currently  no  professional  engineering  occupation  code  or  position  description  for  SE 
within  the  Department  of  Defense  (DOD).  This  makes  it  very  difficult  to  identify  how 
many  systems  engineers  are  within  the  DOD  from  within  this  38,000-member  SPRDE 
workforce. 

A.  BACKGROUND 

1.  Problem  Background 

There  is  currently  no  professional  engineering  occupational  code  or  position 
description  for  SE  within  the  DOD.  Professional  engineering  occupational  codes  are 
used  to  classify  the  characteristics  desired  for  various  engineering  communities,  such  as 
chemical  engineers  or  mechanical  engineers,  for  instance.  Occupational  codes  also 
provide  a  government  job  resource  that  can  assist  in  determining  the  number  of 
employees  in  a  specific  field  or  occupation.  Occupational  codes  can  also  assist  with 
manpower  forecasting  efforts.  Position  descriptions  highlight  the  KSAs  required  to  be 
qualified  for  a  specific  job.  The  position  descriptions  are  helpful  with  finding  the  most 
competent  candidate  for  the  position.  There  is  a  need  for  a  competency  model  offering  a 
set  of  KSAs  that  will  assist  in  creating  position  descriptions  and  a  related  SE  career 
development  plan  designed  specifically  for  systems  engineers  within  the  DON. 

2.  What  Is  Competency 

According  to  (Joshi,  Datta  and  Han  2010),  competency  is  the  ability  to  use  the 
appropriate  KSAs  to  complete  successfully  a  specific  job-related  task.  When  combining 
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competency  with  competence,  it  introduces  competency  assessment.  Competency 
assessment  is  a  tool  found  useful  to  organizations  for  allocating  human  resources  for  a 
successful  employer-employee  match.  Competency  assessment  is  also  beneficial  in 
creating  job-specific  professional  development  and  accurate  training  requirements  for 
employees  to  obtain  a  good  match  for  the  position.  Joshi’s  study  examined  hiring  practice 
trends  through  content  analysis  of  job  advertisements  and  job  types  and  observed  that 
corporations  prefer  to  employ  well-rounded  employees  with  business  knowledge, 
interpersonal  skills  and  technical  skills. 

3.  Competency  Modeling 

Important  to  this  research  was  the  role  of  competency  modeling,  which  is  defined 
as  the  activity  of  determining  the  specific  competencies  that  are  characteristic  of  high 
performance  and  success  in  a  given  job  (LaRocca  n.d.)  A  good  competency  model  has 
the  following  attributes:  it  has  gone  through  much  iteration;  it  focuses  on  a  specific  aspect 
of  competency;  it  is  simple  and  easy  to  understand;  and  it  maps  competencies  across 
levels  (Holt  and  Perry  2011).  The  model  should  map  career  levels  in  a  way  that  is  easy  to 
understand.  For  example,  if  a  given  organization’s  standards  require  that  an  individual 
attain  the  “practitioner”  level  for  a  competency,  then  it  is  assumed  that  the  individual 
must  hold  that  competency  when  at  “supervised  practitioner”  level  also.  In  addition,  a 
good  competency  model  serves  as  a  platform  by  which  individuals  can  assess  their  skill 
set  (Whitcomb,  Khan  and  White  2013). 

B.  PROBLEM  STATEMENT 

Currently,  there  is  no  Position  Category  Description  (PCD)  for  a  systems  engineer 
within  the  Naval  System  Command.  PCDs  have  KSAs  necessary  to  complete  a  job. 
KSAs  for  systems  engineers  within  the  command  vary  depending  on  their  professional 
and  educational  background,  experience,  and  domain  specialization  at  various  career 
levels. 

In  order  to  solve  this  problem,  KSAs  required  to  be  a  competent  systems  engineer 

at  various  career  experience  levels  were  identified.  After  the  KSAs  identification,  the 

KSAs  origin  was  determined.  Does  it  originate  from  undergraduate  education,  on  the  job 
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training  or  professional  development  courses?  Then  each  SE  career  experience  level  was 
identified  and  related  to  the  required  KSAs.  This  effort  resulted  in  a  foundation  to  build  a 
SE  career  development  plan  that  can  be  truly  beneficial  to  both  the  Navel  engineer 
professional  and  the  DOD. 

C.  RESEARCH  OBJECTIVES 

The  objectives  for  this  research  are  to  use  the  COMPOSE  model  that  the  NPS 
Team  created  to  develop  a  competency  model  and  identify  and  evaluate  the  cognitive  and 
affective  domains  associated  with  SE  KSAs.  The  affective  domain  has  to  do  with 
emotions,  feelings  and  attitudes.  Objectives  describe  growth  in  awareness,  attitude, 
emotion,  changes  in  interest,  judgment  and  the  development  of  appreciation. 
Harmonizing  the  COMPOSE  model  with  Bloom’s  Taxonomy  creates  the  baseline 
information  needed  for  the  COMPOSE  while  defining  career  paths  for  systems  engineers. 

D.  RESEARCH  QUESTIONS 

The  following  research  questions  were  developed  to  guide  the  research  and 
address  the  research  objectives.  The  answers  and  conclusions  from  the  analysis  are 
presented  in  Chapters  IV  and  V. 

1.  Primary  Research  Question 

The  primary  research  question  will  help  determine  the  attributes  that  a  naval 
systems  engineering  competency  model  would  need  to  be  useful  to  naval  system 
commands. 

•  Primary  Research  Question  1:  The  Naval  System  Commands  are 
comprised  of  the  Naval  Air  Systems  Command  (NAVAIR),  Naval  Sea 
Systems  Command  (NAVSEA),  Naval  Supply  Systems  Command 
(NAVSUP),  Space  and  Naval  Warfare  Systems  Command  (SPAWAR) 
and  Marine  Corps  Systems  Command  (MARCORSYSCOM).  These  are 
important  because  they  make  up  most  of  the  commands  that  have  much  of 
the  Acquisition  Workforce  for  the  Navy.  The  question  this  research  is  to 
address  is  what  would  a  systems  engineering  competency  model  consist  of 
at  the  general  Naval  Systems  Commands  level  to  use  for  SE  competency 
career  development,  planning  and  tracking? 
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2.  Secondary  Research  Questions 

While  studying  the  attributes  required  to  create  a  useful  Naval  systems 
engineering  competency  model,  a  set  of  additional  secondary  research  questions  surfaced. 

•  Secondary  Research  Question  1:  What  competencies  are  required  for 
Naval  Systems  Engineering? 

•  Secondary  Research  Question  2:  What  KSAs  are  required  for  systems 
engineers  to  develop  for  naval  systems  engineering  competencies  at 
various  career  levels? 

•  Secondary  Research  Question  3:  How  does  the  use  of  Bloom’s  Taxonomy 
for  KSA  definition  relate  to  competency  development  along  a  career  path? 

•  Secondary  Research  Question  4:  Does  the  COMPOSE  model  have 
competencies  accurately  identified  for  the  KSAs  required  to  be  a 
successful  systems  engineer? 

E.  PURPOSE/BENEFIT 

The  benefit  of  the  research  is  that  it  will  provide  a  model,  COMPOSE,  that  can  be 
used  by  several  organizations  to  identify  KSAs  pertinent  to  the  development  of  systems 
engineers.  The  COMPOSE  model  will  also  allow  the  DON  to  formulate  competency 
development  plans  for  the  professional  development  of  systems  engineers.  Finally,  the 
model  can  contribute  to  the  guidance  for  development  of  graduate  and  undergraduate 
curricula  in  systems  engineering. 

F.  SCOPE 

The  objective  of  this  research  is  to  develop  a  competency  model,  and  identify  and 
evaluate  the  cognitive  and  affective  domains  associated  with  SE  KSAs.  The  scope  for  this 
project  includes  defining  KSAs  using  Bloom’s  Taxonomy  Cognitive  and  Affective 
Domains,  harmonizing  the  COMPOSE  model  with  Krathwohl’s  revision  of  Bloom’s 
Taxonomy,  developing  an  approach  and  methodology  to  obtain  baseline  information 
needed  for  the  COMPOSE  model  life  cycle  development  (knowledge,  skills,  abilities  and 
behaviors,  and  related  education,  training,  and  experience  needed),  and  defining  career 
paths  for  systems  engineers  Gobs,  assignments,  and  timing)  for  the  DOD. 
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G.  METHODOLOGY 


The  team  identified  eight  SE  competency  models  to  determine  the  potential  SE 
competencies  for  the  Navy  and  organized  the  elements  based  on  their  similarities.  The 
competency  models  used  for  the  foundation  were  the  SE  Workforce  Development  from 
the  NUWC,  INCOSE  UK,  NASA,  DAU  SPRDE  and  Boeing.  Each  entry  from  the  five 
competency  models  were  analyzed  and  re-organized  based  on  the  similarity  of  their 
competency  definitions.  Each  of  the  models  had  different  approaches  and  formats.  Some 
formats  combined  relevant  SE  competencies  with  generic  engineering  competencies.  The 
KSAs  were  realigned  to  fit  the  COMPOSE  model  by  first  eliminating  duplication,  then 
eliminating  items  that  didn’t  seem  explicitly  defined  as  a  relevant  SE  competency,  re¬ 
organizing,  and  lastly  by  updating  the  COMPOSE  model  to  incorporate  changes  in  the 
2013  DAU/SPRDE  Competency  model  update.  Once  the  models  are  harmonized  into  a 
single  coherent  model,  it  will  be  analyzed  from  various  perspectives  to  study  the 
characteristics  in  order  to  understand  how  it  would  be  useful  as  a  baseline  model  for 
ultimate  use  in  a  future  model  validation  process. 

H.  THESIS  STATEMENT 

This  study  will  analyze  and  determine  the  KSAs  required  for  DOD  systems 
engineers  to  be  considered  competent  systems  engineers  at  various  levels  of  career 
experience  levels.  Following  Bloom’s  Taxonomy  cognitive  and  affective  domain 
approach  provides  a  comprehensive  model.  This  SE  competency  model  will  provide  the 
KSAs  that  will  assist  in  creating  position  descriptions  and  a  related  SE  career 
development  plan  designed  specifically  for  systems  engineers  within  the  DOD. 
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Figure  1.  Bloom’s  Taxonomy  Domain 
(from  Wrightstuffmusic  2014) 


I.  REPORT  ORGANIZATION 

Chapter  I  comprises  the  background,  research  objectives  and  questions;  Chapter  II 
focuses  on  the  analysis  portion  of  this  research.  Chapter  III.  reveals  the  data  and 
methodology  behind  the  analysis.  Chapter  IV  discusses  the  results  of  the  analysis. 
Chapter  V.  shares  the  conclusions  and  suggestions  for  further  research  efforts. 

J.  CHAPTER  SUMMARY 

This  chapter  reviewed  the  evolution  of  the  COMPOSE  model  by  identifying  the 
eight  original  SE  competency  models  used  to  create  the  model.  The  problem  statement 
was  stated,  research  objectives  and  research  questions  were  established,  the  importance 
of  this  study  was  explained  and  the  problem  has  been  examined.  To  conclude  this  chapter 
the  scope  and  methodology  were  explained  and  the  thesis  statement  defined.  Next  is  a 
discussion  on  the  tools  used  in  this  analysis  to  include  the  literature  review  and  Bloom’s 
Taxonomy. 
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II.  LITERATURE  REVIEW 


The  Taxonomy  of  Educational  Objectives  is  a  scheme  for  classifying 
educational  goals,  objectives,  and,  most  recently,  standards.  It  provides  an 
organizational  structure  that  gives  a  commonly  understood  meaning  to 
objectives  classified  in  one  of  its  categories,  thereby  enhancing 
communication. 

— David  Krathwohl 
A  Revision  of  Bloom ’s  Taxonomy:  An  Overview 

This  chapter  will  focus  on  current  literature  used  in  the  analysis.  First  is 
information  on  Bloom’s  Taxonomy,  including  what  it  is  and  how  it  is  useful  in 
competency  development.  Next  is  a  summary  of  successful  competency  models. 

A.  BLOOM’S  TAXONOMY 

According  to  Bloom,  skills  can  be  categorized  as  cognitive,  affective,  or 
psychomotor.  They  are  learned  and  acquired  through  education  training  and  experience 
(Bloom,  Engelhart,  Furst,  Hill  and  Krathwohl  1956).  Skills  are  a  composite  of  abilities, 
techniques,  and  knowledge.  They  are  developmental,  incremental,  and  reference-based 
for  a  desired  application,  in  this  case,  systems  engineering  (Whitcomb,  Khan  and  White 
2013). 

Benjamin  Bloom’s  system  of  classification  categorizes  learning  behavior  into  six 
levels  of  cognitive  complexity  that  are  cumulative  and  hierarchical.  These  levels 
differentiate  three  cognitive  domains  of  learning  (knowledge,  how  it  is  learned), 
psychomotor  (skills,  doing,  hand/body),  and  affective  (attitudes,  feelings,  appreciation 
and  value).  Each  level  of  the  taxonomy  is  associated  with  action  verbs  that  can  be  used  to 
construct  learning  outcomes  and  objectives  in  the  cognitive  domain.  Bloom’s  Taxonomy 
was  created  in  1956  by  a  group  of  educational  psychologists  led  by  Benjamin  Bloom. 
The  goal  of  the  group  was  to  create  a  foundation  for  curriculum  design  and  to  ensure  that 
educational  objectives  were  being  met  (Bloom,  Engelhart,  Furst,  Hill  and  Krathwohl 
1956). 
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The  original  version  of  Bloom’s  Taxonomy  was  created  more  than  50  years  ago 
and  many  professionals  have  used  it  and  built  upon  it  effectively.  The  International 
Council  on  Systems  Engineering  (INCOSE)  Graduate  Reference  Curriculum  for  SE 
(GRCSE)  used  Bloom’s  Taxonomy  as  a  foundation  with  a  heavy  focus  in  the  cognitive 
domain  and  a  smaller  emphasis  on  the  affective.  Krathwohl  and  Anderson  published  a 
revised  Bloom’s  Taxonomy  update  in  2001.  This  updated  version  resembles  the  six 
original;  however,  “Knowledge”  is  re-categorized  as  “Remember,”  “Comprehension”  is 
“Understand,”  and  “Synthesis”  is  “Create.”  Figure  5  illustrates  the  changes  between  the 
1956  and  2001  versions  of  Bloom’s. 


Changes  to  Bloom’s 


1956 


2001 


Comprehension 


Evaluate 


Understand 


Knowledge 


1 


Remember 


Noun 


to  Verb  Form 


Figure  2.  Changes  to  Bloom’ 

(from  Wilson  2013) 

Initially,  the  1956  version  of  Bloom’s  was  used  to  develop  the  COMPOSE  model 
because  it  was  used  in  GRCSE.  The  COMPOSE  model  has  gone  through  several 
iterations.  The  most  current  version  of  the  COMPOSE  model  uses  Krathwohl’ s  revision 
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of  Bloom’s  structure.  The  COMPOSE  model  only  evaluates  the  cognitive  and  affective 
domains.  The  NPS  Team  assumed  that  the  psychomotor  is  not  relevant  in  this  study 
because  it  does  not  apply  to  engineering. 

The  cognitive  domain  includes  knowledge,  critical  thinking  and  the  development 
of  intellectual  skills.  Originally,  the  categories  in  the  cognitive  domain  were  arranged  by 
order  of  complexity  with  the  following  Bloom’s  Levels:  knowledge,  comprehension, 
application,  analysis,  synthesis,  and  evaluation.  Table  2  shows  the  cognitive  domains. 


Level 

Sub-Level 

Competency 

Outcome  Descriptors 

Knowledge 

•  Knowledge  of  specifics 

•  Knowledge  of  terminology 

•  Knowledge  of  specific  facts 

•  Knowledge  of  ways  and 
means  of  dealing  with 
specifics  (processes) 

•  Knowledge  of  the 
universals  and  abstractions 

Ability  to  remember  previously  learned 
materi a  1 .  Test  observation  and  reca  1 1  of 
information;  i.e.,  "bring  to  mind  the 
appropriate  information;"  e.g.,  dates, 
events,  places,  knowledge  of  major  ideas, 
and  mastery  of  subject  matter. 

List,  define,  tell,  describe, 
identify,  show,  label, 
collect,  examine, 
tabulate,  quote,  and 
name  (who,  when,  where, 
etc.). 

Comprehension 

•  Translation 

•  Interpretation 

•  Extrapolation 

Ability  to  understand  information  and 
ability  to  grasp  meaning  of  material 
presented;  e.g.,  tra nsl ate  knowl edge  i nto 
new  context,  interpret  facts,  compare, 
contrast,  order,  group,  infer  causes, 
predict  consequences,  etc. 

Summarize,  describe, 
interpret,  contrast, 
predict,  associate, 
distinguish,  estimate, 
differentiate,  discuss,  and 

extend. 

Application 

•  Application  of  methods 
and  tools 

•  Use  of  common 

techniques  and  best 
practices 

Ability  to  use  learned  material  in  new  and 
concrete  situations;  e.g.,  use  information, 
methods,  concepts,  and  theories  to  solve 
problems  requiring  the  skills  or  knowledge 
presented. 

Apply,  demonstrate, 
calculate,  complete, 
illustrate,  show,  solve, 
examine,  modify,  relate, 
change,  classify. 

Analysis 

•  Analysis  of  elements 

•  Analysis  of  relationships 

•  Analysis  of  organizational 
principles 

Ability  to  decompose  learned  material  into 
constituent  parts  in  order  to  understand 
structure  of  the  whol e.  This  i  ncl  udes 
seei ng  patterns,  organization  of  parts, 
recognition  of  hidden  mea ni ngs,  and 
obviously,  identification  of  parts. 

Analyze,  separate,  order, 
explain,  connect,  classify, 
arrange,  divide,  compare, 
select,  explain,  and  infer. 

Synthesis 

•  Production  of  a  unique 

communication 

•  Production  of  a  plan,  or 
proposed  set  of  operations 

•  Derivation  of  a  set  of 

abstract  relations 

Abi  1  ity  to  put  pa  rts  together  to  form  a  new 
whole.  This  i nvolves  the  use  of  existi ng 
ideas  to  create  new  ones,  generalizing 
from  facts,  relati ng  knowledge  from 
several  areas,  and  predicting  and  drawing 
conclusions.  It  may  also  involve  the 
adaptation  of  "general"  solution 
pri  nciples  to  the  embodi  ment  of  a  specific 
problem. 

Combine,  integrate, 
modify,  rearrange, 
substitute,  plan,  create, 
design,  invent,  what-if 
analysis,  compose, 
formulate,  prepare, 
generalize,  and  rewrite. 

Evaluation 

•  Judgements  in  terms  of 
internal  evidence 

•  Judgments  in  terms  of 
external  criteria 

Abi  1  ity  to  pass  j udgment  on  va  1  ue  of 
materi  a  1  withi  n  a  given  context  or 
purpose.  This  involves  making 
comparisons  and  discriminating  between 
ideas,  assessing  the  val  ue  of  theories, 
making  choices  based  on  reasoned 
arguments,  verifying  the  value  of  evidence, 
and  recognizing  subjectivity. 

award,  choose,  conclude, 
criticize,  decide,  defend, 
determine,  dispute, 
evaluate,  judge,  justify, 
measure,  compare,  mark, 
rate,  recommend,  rule  on, 
select,  agree,  interpret, 
explain,  appraise, 
prioritize,  opinion. 

Table  1.  Bloom’s  Taxonomy  of  Educational  Outcomes — Cognitive  Domain 

(from  GRCSE  2011,  97-98) 
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The  affective  domain  has  to  do  with  emotions,  feelings  and  attitudes.  Objectives 
describe  growth  in  awareness,  attitude,  emotion,  changes  in  interest,  judgment  and  the 
development  of  appreciation.  Table  3  shows  the  affective  domains.  While  most  educators 
do  not  include  the  affective  domain  in  their  curriculum,  in  fact  this  domain  is  completely 
absent  from  the  DAU  SPRDE  curriculum,  these  outcomes  are  especially  critical  to  the 
success  of  systems  engineers  because  it  is  also  important  to  communicate  well  and  work 
cohesively  as  part  of  a  team  (Hudson  2013). 


Level 

Sub-Level 

Competency 

Outcome  Descriptors 

Receiving 

•  Awareness 

•  Willingness  to  receive 

•  Controlled  or  selected 

attention 

The  learner  is  aware  of  stimuli 
and  is  willing  to  attend  to 
them.  The  learner  may  be  able 
to  control  attention  to  the 

stimuli. 

Focuses  on  and  is  aware  of 
aesthetics,  focuses  on  human 
values,  is  alert  to  desirable 
qualities,  and  shows  careful 
attendance  to  input. 

Responding 

•  Acquiescence  in 
responding 

•  Willingness  to  respond 

•  Satisfaction  in  response 

The  learner  makes  a  conscious 
responseto  the  stimuli  related 
to  the  aesthetic  or  quality.  At 
this  level  the  learner  expresses 
an  interest  in  the  aesthetic 
things. 

Demonstrates  willing  compliance 
and  obedience  to  regulations  and 
rules,  seeks  broad-based 
information  to  act  upon,  and 
accepts  responsibility  and 
expresses  pleasure  for  own 

situation. 

Valuing 

•  Acceptance  of  a  value 

•  Preference  for  a  value 

•  Commitment 

The  learner  recognizes  worth  in 
the  subject  matter. 

Continuing  desire  to  achieve, 
assumes  responsibility  for,  seeks  to 
form  a  view  on  controversial 
matters,  devotion  to  principles,  and 
faith  in  effectiveness  of  reason. 

Organization 

•  Conceptualizatin  of  a  value 

•  Organizationof  a  value 
system 

The  learner  is  able  to  organize 
a  number  of  values  into  a 
system  of  values  and  can 
determine  the  inter¬ 
relationships  of  the  values. 

Identifies  characteristics  of  an 
aesthetic,  forms  value-based 
judgments,  and  weighs  alternative 
policies. 

Characterization 

•  Generalilzed  set 

•  Characterization 

The  learner  acts  consistently 
with  the  systems  of  attitudes 
and  values  they  have 
developed.  The  values  and 
views  are  integrated  into  a 
coherent  worldview. 

Readiness  to  revise  judgment  in 
light  of  evidence,  judges  problems 
and  issues  on  their  merit  (not 
recited  positions),  and  develops  a 
consistent  philosophy  of  life. 

Table  2.  Bloom’s  Taxonomy  of  Educational  Outcomes — Affective  Domain 

(from  GRCSE  2011,  99-100) 


Systems  engineers  lead  systems  projects,  negotiate  outcomes  with  a  diverse  group 
of  stakeholders,  make  value  judgments  and  must  have  the  ability  to  deliberately  take  the 
systems  perspective  (GRCSE,  2011,  p.  95).  Leading,  negotiating,  and  having  sound 
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judgment  are  all  affective  outcomes.  Almost  29  percent  of  the  COMPOSE  model’s  KSAs 
fall  into  the  affective  domain  which  indicates  how  important  both  technical/technical 
management  and  professional  skills  are  in  becoming  a  successful  systems  engineer.  The 
five  levels  of  the  affective  domain  are  arranged  by  order  of  complexity  as  follows: 
receiving,  responding,  valuing,  organization  and  characterization.  Table  3  shows  the 
affective  domains.  The  cognitive  and  affective  skills  are  mapped  to  proficiencies  within 
the  competencies  based  on  the  key  words  used  which  correlate  to  a  specific  cognitive  or 
affective  category.  For  example,  if  a  KSA  says  “Analyze  sponsor/customer  information 
for  the  development  of  acquisition  plans”  The  verb  “analyze”  corresponds  to  the  higher- 
level  cognitive  domain  as  shown  in  Tables  2-3. 

The  current  version  of  the  COMPOSE  model  uses  Krathwohl’s  version  of 
Bloom’s  Taxonomy.  Krathwohl  converts  the  category  from  a  noun  to  a  verb.  For 
example  within  the  cognitive  domain  there  was  originally  “Evaluation”,  in  Krathwohl’ s 
version  it  is  “Evaluate”.  Figure  6  shows  Krathwohl’s  Bloom’s  Taxonomy  Update. 
Analysis  reveals  that  about  67  percent  of  the  COMPOSE  model’s  KSAs  fall  into  the 
cognitive  domain  and  33  percent  are  aligned  with  the  affective  domain  (Whitcomb,  Khan 
and  White  2013). 


Figure  3.  Krathwohl’s  Bloom’s  Taxonomy  Update — Cognitive  Domain 
(from  Florida  International  University  2014) 
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B.  SUCCESSFUL  COMPETENCY  MODELING  APPROACHES 

Prior  to  developing  COMPOSE,  several  competency  model  approaches  were 
explored,  such  as  The  Pragmatic  Guide  to  Competency  report  by  Holt  and  Perry,  The 
Career  and  Competency  Pathing  report  by  LaRocca,  the  GRCSE  report  by  INCOSE  and 
The  U.S.  Department  of  Labor  Employment  and  Training  Administration  (ETA)  User 
Guide  to  Competency  models.  The  COMPOSE  model  was  developed  with  all  of  these 
aspects  in  mind  (Alexander  2013). 

The  Holt  and  Perry  Guide  focuses  on  what  defining  a  good  competency  model. 
As  discussed  in  Chapter  I,  this  guide  concludes  that  a  good  competency  model  goes 
through  many  iterations,  focuses  on  a  specific  aspect  of  competency  but  is  easy  to 
understand,  maps  competencies  across  levels,  keeps  a  small  number  of  levels,  maps 
levels  clearly  and  emphasizes  technical  skills  (Holt  and  Perry  2011). 

Successful  competency  models  also  serve  as  a  platform  by  which  individuals  can 
assess  their  skill  set.  Although,  not  in  the  scope  of  this  research,  in  the  future  the 
COMPOSE  model  will  be  used  in  efforts  to  assess  individual’s  skills.  An  example  of 
this,  following  the  Body  of  Knowledge  model  established  by  the  Association  of  Project 
Management  (APM),  is  to  apply  a  rating  (which  may  be  1,2,  3,  or  5  -no.  4)  that  provides 
a  weight  for  making  the  calculation  to  derive  a  point  value,  by  which  an  individual  can 
assess  how  competent  he  or  she  is  in  an  activity.  To  do  so,  an  individual  can  determine 
how  valuable  certain  information  on  a  learned  activity  has  been  by  defining  the  value  on 
a  scale  of  2  to  10,  where  “2”  is  considered  of  “little  value”  and  “10”  is  considered  to  be  of 
“high”  value.  The  individual  would  then  multiply  the  rating  and  the  value  together  to 
attain  a  “point  value”  -  which  can  then  be  assessed  to  determine  evidence  of  competency. 
Although  out  of  scope  for  this  research  effort,  it  should  be  stressed  that  along  with  having 
an  assessment  procedure  in  a  competency  model,  the  competency  model  should  also 
include  a  framework  for  validating  assessments. 

The  Career  and  Competency  Pathing  Competency  Modeling  Approach  by 
LaRocca  concentrates  on  how  organizations  can  identify  their  core  competencies  and 
how  they  can  apply  the  competency  data  to  improve  performance.  Additionally,  it 
explains  some  emerging  trends  in  competency  modeling.  According  to  LaRocca,  it  is 
imperative  that  organizations  understand  what  knowledge,  skills  and  abilities  are  required 
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for  people  in  key  roles  to  deliver  business  goals.  LaRocca  also  stresses  there  are  six 
stages  in  defining  a  competency  model  for  a  given  job  role  which  include  the  following: 
defining  the  criteria  for  superior  performance  in  a  given  role,  choosing  a  sample  of 
people  performing  the  role  for  data  collection,  collecting  sample  data  about  behaviors 
that  lead  to  success,  developing  hypotheses  about  the  competencies  of  outstanding 
performers  and  how  competencies  work  together  to  produce  desired  results,  validating 
results  of  data  collection  and  analysis  and  applying  the  competency  models  in  real  case 
scenarios  (LaRocca  n.d.). 

Key  findings  from  the  Schoonover  Associates  and  Arthur  Anderson  2002  study 
determining  how  organizations  actually  use  competency  data  to  be  successful  in  real-life 
applications  were  also  highlighted  (Boulter,  Dalziel  and  Hill  1998).  Of  these,  it  was  noted 
that  the  use  of  competencies,  in  order  of  their  effectiveness,  includes  hiring,  job 
descriptions,  training,  performance  management,  and  development  planning  and  career 
pathing.  (Shoonover,  Shoonover,  Nemerov  and  Ehly  2012).  While  not  in  the  scope  of  this 
research,  it  would  be  interesting  to  analyze  exactly  how  the  COMPOSE  model  was  used 
in  a  given  organization  after  being  adopted.  For  example,  what  percentage  of  the  model 
was  used  to  create  job  descriptions,  initiate  training  requirements? 

GRCSE  is  a  part  of  the  Body  of  Knowledge  and  Curriculum  to  Advance  System 
Engineering  (BKCASE)  (INCOSE  n.d.).  This  approach  keys  in  on  how  to  use  Bloom’s 
Taxonomy  to  set  the  level  of  attainment  of  educational  or  learning  outcomes  required  for 
students  engaged  in  an  educational  unit,  course  or  program.  In  GRCSE,  the  major  focus 
is  on  the  cognitive  domain,  which  is  concerned  with  knowledge  and  how  it  is  learned 
(Huitt  n.d.).  The  affective  domain  is  a  minor  focus,  as  it  is  concerned  with  feelings, 
appreciation  and  valuation  of  the  content  that  is  learned.  In  some  education,  for  example 
military  and  theological,  the  affective  domain  is  an  explicit  focus  of  the  outcomes 
because  of  the  high  standard  of  morals  and  values  emphasized.  This  is  why  in  this  study 
focuses  on  both  the  cognitive  and  affective  domains. 

The  U.S.  Department  of  Labor,  Employment  and  Training  Administration  (ETA) 
User  Guide  to  Competency  Model  (U.S.  Department  of  Labor  Employment  and  Training 
Administration  2013)  shows  five  steps  to  developing  a  competency  model  as  shown  in 
Figure  7. 
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STEP  1:  Gather  background  information 


Existing  frameworks  and  models  are  analyzed,  organized  and  evaluated 

to  determine  affinities 


STEP  2:  Develop  a  draft  competency  model  framework 


Themes  and  patterns  in  the  existing  information  are  identified, 
reorganized  and  a  draft  model  is  developed. 


STEP  3:  Gather  feedback  from  SMEs 


Review  is  requested  to  determine  if  the  framework  reflects  appropriate 
competencies;  if  any  competencies  are  missing  and  if  any  terminology 

changes  need  to  be  made. 


wA 


STEP  4:  Refine  the  competency  framework 


Development  of  a  competency  model  is  an  iterative  process.  Revisions, 
additions,  deletions,  and  reorganization  occur  at  this  step. 


STEP  5:  Validate 


Ensure  acceptance  by  a  target  community  of  users. 


Figure  4.  Competency  Model  Development  Steps  (from  U.S.  Department  of  Labor 

Employment  and  Training  Administration  2013) 


Note  that  steps  3  and  4  are  repeated  until  the  subject  matter  experts  (SMEs)  and 
the  development  team  agrees  that  the  model  is  an  all-inclusive  representation  of  required 
KSAs. 

C.  EXISTING  SE  COMPETENCIES  SERVE  AS  A  FOUNDATION 

The  last  thing  that  the  world  needs  is  yet  more  frameworks,  so,  again,  the 
idea  of  cherry-picking  different  parts  from  different  frameworks  is  a  very 
attractive  one. 


— John  Holt  and  Simon  A.  Perry 
A  Pragmatic  Guide  to  Competency:  Tools,  Frameworks  and  Assessment 
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Several  competency  models  were  used  to  construct  the  COMPOSE  model 
including  NUWC,  INCOSE  UK,  NAVAIR,  SPA  WAR,  Boeing,  NASA,  MITRE  and 
SPRDE.  Collectively,  these  competency  models  created  the  forty-one  competencies  in 
the  COMPOSE  model.  Figure  8.0  is  an  illustration  of  how  existing  competency  models 
were  used  to  create  the  COMPOSE  model. 


INCOSE  UK 
SE  Competency  Model 


.Mr1  - - -  ■  - 

COMPOSE 

Model 


NAVAIR  SE 
Competency  Model 


NASA 

^  oemg  SE  Competency  Model 

SE  Competency  Model 


MITRE  SE 


Competency 

Model 


NUWC  Newport 
SE  Workforce 
Development  Model 


SPAWAR  SE 
Competency  Model 


DAU  SPRDE 
SE  Competency  Model 


Figure  5.  Competency  Sources  Used  in  COMPOSE 
(  from  Whitcomb,  Khan  and  White  2013) 

All  of  the  models  introduced  were  used  as  a  foundation  to  identify  competencies 
to  map  to  the  41  competencies  for  the  COMPOSE  model.  The  next  step  in  developing  the 
COMPOSE  model  was  to  identify  KSAs  and  map  them  to  the  41  competencies. 
However,  only  three  of  these  models  NUWC,  INCOSE  UK  and  MITRE  were  used  to 
derive  the  KSAs  for  the  COMPOSE  model  based  on  their  format  and  range  of  KSAs  for 
SE.  Additionally,  the  DAU  SPRDE  Level  I,  II,  and  III  course  learning  objectives  were 
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transformed  into  KSAs  and  added  to  the  model.  Figure  9  shows  the  models  used  to  derive 
the  KSAs.  The  Overall  COMPOSE  model  v0.78  has  2,914  KSAs. 


D 


COMPOSE 

Model 


INCOSE  UK 
SE  Competency  Model 


NUWC  Newport 
SE  Workforce 
Development  Model 


Jl 


o 


MURE 

SE  Competency'  Model 


SPRDE  Level  I-m 
Course 

Learning  Objectives 


Figure  6.  KSAs  sources  used  in  COMPOSE 

(from  Whitcomb,  Khan  and  White  2013) 

1.  SE  Workforce  Development  Naval  Underwater  Warfare  Center 
Competency  Model 

The  purpose  of  the  Naval  Underwater  Warfare  Center  (NUWC)  SE  workforce 
competency  model  is  to  articulate  clearly  and  establish  the  development  of  the  NUWC 
SE  workforce.  With  twenty-eight  competencies,  six  experience  levels  and  689  KSAs,  this 
competency  model  displays  a  series  of  steps  that  are  used  in  sequence  to  derive 
requirements  and  then  transform  those  into  solutions.  The  model  is  written  in  such  a 
manner  that  it  is  intended  to  be  used  without  the  size  or  complexity  of  the  problem  being 
an  issue  (Walter  2013).  The  goal  for  the  NUWC  competency  model  is  to  provide  a  tool 
for  the  SE  Workforce  that  will  serve  as  a  guide  to  ensure  that  the  program  execution  is 
right  from  the  beginning  and  that  problems  are  addressed  early.  If  used  correctly,  it  could 
aid  in  minimizing  risk  and  reducing  costs.  The  model  clearly  identifies  the  KSAs, 
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behaviors,  attitudes,  attributes  and  performance  expected  of  a  SE  at  specified  career 
experience  levels.  However,  there  is  no  explanation  of  where  or  how  these  KSAs  can  be 
obtained  if  a  SE  is  lacking  those  KSAs. 

2.  International  Council  on  Systems  Engineering  United  Kingdom 
Competency  Model 

This  system  engineering  competency  model  was  developed  by  a  working  group  of 
SE  representatives  from  over  10  different  organizations.  It  is  a  framework  for  generic 
purpose  and  organized  in  a  way  that  is  easy  to  tailor  for  specific  purposes.  The  model’s 
foundation  is  built  on  official  system  engineering  standards,  and  it  focuses  primarily  on 
technical  SE  competencies,  while  making  it  evident  that  when  linked  with  processes, 
organizations  and  infrastructure  then  capabilities  are  created.  This  model  is  composed  of 
21  competencies,  four  experience  levels  and  273  KSAs.  Similar  to  other  SE  Competency 
models,  the  International  Council  on  Systems  Engineering  (INCOSE)  model  identifies 
key  competencies  in  SE,  introduces  supporting  techniques,  identifies  basic  skills  and 
behaviors  and  describes  the  domain  knowledge  required  (UK,  INCOSE). 

3.  MITRE  Competency  Model 

Similar  to  the  COMPOSE  model’s  methodology,  the  MITRE  Competency  model 
takes  a  government  view  and  approach  to  systems  engineering.  This  comprehensive 
model  identifies  all  of  the  competencies  needed  to  fulfill  a  particular  role  in  the  SE 
process;  for  example,  Systems  Engineering  Planning  (Transformational  Planning, 
Government  Acquisition  Support,  Contractor  Evaluation,  Risk  Management, 
Configuration  Management,  Integrated  Logistics  Support,  Q&A  Measurement  and 
Continuous  Process  Improvement)  and  the  Management  or  Systems  Engineering  Life 
Cycle  (Concept  of  Operation,  Requirements,  Architecture,  System  Design,  Systems 
Integration,  Test  and  Evaluation,  Systems  Implementation,  O&M  and  Transition)  are 
examples  of  the  competencies  MITRE  identified  that  fulfill  a  specific  role  in  the  SE 
process.  This  approach  to  competency  modeling  is  more  useful  in  developing  individuals 
and  teams  for  a  wide  range  of  system  engineering  jobs.  The  non-technical  competencies 
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were  extracted  from  the  MITRE  Institute  Leadership  and  Management  Competency 
models.  The  model  is  composed  of  forty-one  competencies  and  1,262  KSAs. 

4.  Defense  Acquisition  University  System  Planning,  Research, 
Development  and  Engineering  Learning  Objectives 

Using  a  competency-based  model,  Defense  Acquisition  University  (DAU)  used 
Bloom’s  taxonomy  to  provide  the  framework  for  course  development  (Layton  2007). 
Level  I  certifications  were  structured  to  reflect  basic  knowledge,  Level  II  built  on  that 
basic  knowledge  introducing  practical  application  and  small  group  scenarios,  and  Level 
III  certifications  were  to  develop  synthesis  and  evaluation  abilities.  The  112  course 
learning/  performance  objectives  (CL/PO)  and  542  educational  learning  objectives  (ELO) 
were  categorized  according  to  the  Graduate  Reference  Curriculum  for  Systems 
Engineering  (GRCSE)  version  of  Bloom’s  Taxonomy.  None  of  the  data  fell  into  the 
affective  domain;  hence,  all  CL/POs  and  ELOs  were  categorized  into  the  cognitive 
domain  as  knowledge,  comprehension,  application,  analysis,  synthesis  or  evaluation. 

D.  CHAPTER  SUMMARY 

This  chapter  discussed  in  detail  all  of  the  frameworks  that  contributed  to  the 
creation  of  the  COMPOSE  model.  Six  references  and  eight  competency  models  were 
used  to  investigate  Bloom’s  taxonomy,  competency  model  definitions,  and  the 
application  of  the  competency  models  for  this  research.  Bloom’s  Taxonomy  was  used  to 
categorize  each  KSAs  into  the  cognitive  and  affective  domain.  NUWC,  INCOSE  UK, 
NAVAIR,  SPAWAR,  NASA,  MITRE,  SPRDE  and  Boeing  competency  models  serve  as 
the  foundation  for  the  COMPOSE  model. 


18 


III.  COMPETENCY  MODEL  DEVELOPMENT 


Chapter  II  provided  more  information  on  how  Bloom’s  Taxonomy  is  useful  in 
competency  development  and  summarize  the  existing  competency  models  that  were 
chosen  to  be  the  foundation  for  the  COMPOSE  model.  This  chapter  will  discuss  in  detail 
how  the  COMPOSE  model  evolved.  One  of  the  goals  of  the  NPS  team  was  to  have  a 
model  that  could  go  through  several  updates  and  changes;  therefore,  the  NPS  Team 
dedicated  a  tremendous  amount  of  effort  on  aligning  information  from  the  existing 
competency  models  to  fit  the  COMPOSE  model,  eliminating  data  that  are  not  useful, 
incorporating  the  DAU  SPRDE  CL/POs  and  ELOs  and  identifying  training  levels.  KSAs 
were  mapped  to  fit  GRCSE  Bloom’s  Taxonomy  in  the  first  version  of  the  COMPOSE 
model.  In  the  most  current  version  of  the  COMPOSE  model  KSAs  are  mapped  to 
Krathwohl’s  revised  version  of  Bloom’s  Taxonomy. 

The  COMPOSE  model  contains  an  aggregate  of  the  core  technical/technical 
management  and  professional  KSAs  researched  from  existing  competency  models  from 
various  naval  engineering  enterprises.  The  KSAs  described  in  the  COMPOSE  model  are 
those  that  are  vital  for  the  development  of  proficient  systems  engineers.  The  COMPOSE 
model  provides  a  basis  for  a  single,  coherent  SE  competency  model  that  will  assist  in 
creating  position  descriptions  and  a  related  SE  career  development  plan  designed 
specifically  for  systems  engineers  within  the  DON,  and  to  provide  a  basis  for  a  model  for 
DOD.  Table  1  identifies  the  attributes  of  the  COMPOSE  model  and  compares  them  to 
what  the  Pragmatic  Guide  to  Competency  states  a  good  competency  model  should 
contain. 

The  COMPOSE  model  was  envisioned  to  focus  on  the  specific  competencies  that 
define  systems  engineers  on  a  primarily  technical  and  technical  management  basis;  for 
example,  Mission-Level  Assessment,  Requirements  Analysis,  or  Architecture  Design  are 
all  technical  and  technical  management  competencies.  The  COMPOSE  model  also 
includes  generic  engineering  professional  skills;  such  as  Coaching  and  Mentoring, 
Communication,  and  Personal  Effectiveness/Peer  Interaction. 


19 


Appendix  A  shows  an  example  of  the  competencies  mapped  across  experience  and 
proficiency  levels.  The  tables  also  show  the  proficiency  levels  within  each  competency 
based  on  the  analysis.  The  team  categorized  KSAs  in  the  form  of  Bloom’s  Taxonomy, 
primarily  into  either  the  cognitive  or  affective  domain,  for  each  KSAs  for  core  (technical) 
SE  competencies  to  determine  the  proficiency  levels  within  the  SE  career  development 
levels. 

The  COMPOSE  model  can  be  used  to  tailor  sets  of  KSAs  desired  to  develop 
competent  system  engineers  for  any  DON  organization.  The  model  is  also  intended  to  be 
used  by  universities  and  training  organizations  to  inform  their  development  of  learning 
objectives  to  meet  the  appropriate  competency  attainment  for  various  SE  positions. 
Additionally,  the  model  should  be  useful  as  a  foundation  for  defining  systems 
engineering  professional  certification. 


Attributes  ot  a  Good  Competency 
Model 

Model  Meets 
Requirement 
(Y/N) 

PDA 

1.  Many  Iterations 

Y 

2.  Simple  and  easy  to  understand 

Y 

3.  Maps  levels  in  a  non-complex  format 

Y 

4.  Emphasizes  technical  skills  versus  soft 

NPS  Team  is  currently  looking  at  the  leadership, 
problem  solving,  professional  ethics,  interpersonal 
and  personal  skills,  strategic  thinking  and 
communications  separately  to  determine  whether 
there  are  any  key  "soft"  competencies  systems 

skills 

N 

engineers  should  exhibit. 

5.  Provides  an  assessment  procedure  and  a 

For  Future  Research.  Plan  to  work  with  OPM  to 

framework  for  validating  assessments 

N 

validate  the  model. 

6.  Combines  skills  from  different  models  to 
generate  a  complete  scope  of 
competencies  required  for  a  systems 
engineer 

Y 

Key: 

SE-  Systems  Engineering 
Y/N-YesorNo 
POA-  Plan  of  Action 

Table  3.  COMPOSE  Attributes  (Adapted  from  Holt  and  Perry  2011) 
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1. 


Evolution  of  the  COMPOSE  Model 


There  is  consensus  between  the  Naval  Postgraduate  School  (NPS)  systems 
engineering  (SE)  research  team  and  the  Deputy  Assistant  Secretary  of  the  Navy  (DASN), 
Research,  Development,  Test  and  Evaluation  (RDT&E),  Chief  Engineer  (CHENG)  that 
the  required  competencies  and  certifications  for  proficient  DON  systems  engineers  need 
to  be  examined  (Whitcomb,  Khan  and  White  2013).  The  DASN  (RDT&E)  desires  a 
concrete  study  of  what  systems  engineers  do  and  how  they  develop  throughout  their 
careers.  As  part  of  a  DASN  (RDT&E)  strategic  initiative,  a  team  from  NPS’s  Systems 
Engineering  Department  developed  an  overall  approach  in  order  to  design  a  naval 
systems  engineering  competency  model,  ultimately  named  COMPOSE 

The  NPS  model  began  with  the  identification  of  eight  previously  existing  SE 
competency  models. 

•  NUWC 

•  INCOSE  UK 

•  MITRE 

•  NASA 

•  NAVAIR 

•  SPAWAR 

•  DAU  SPRDE 

•  Boeing 

Each  of  the  SE  competency  models  used  is  described  in  more  detail  in  Chapter  II. 
Elements  of  these  eight  models  were  categorized  based  on  similarity,  with  their 
competencies  mapped  to  the  original  29  DAU/SPRDE  competencies  and  duplicate 
elements  were  eliminated.  The  COMPOSE  model  used  several  other  SE  competency 
models  as  a  foundation  to  get  a  broad  perspective  of  the  KSAs  required  to  be  a  competent 
SE.  The  COMPOSE  model  also  includes  the  DAU  SPRDE-SE  learning  objectives,  the 
degree  to  which  the  learning  objectives  reflect  actual  competencies  needed  to  perform  as 
an  acquisition  systems  engineer,  the  impact  they  have  or  potentially  have  on  SPRDE 
acquisition  workforce  members,  and  how  various  members  fit  into  the  current  DAU 
certification  structure. 
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Skills,  abilities,  and  behaviors,  were  combined  with  knowledge  to  form  the  basis 
for  determining  how  well  someone  can  perform  as  a  systems  engineer  (Whitcomb,  Khan 
and  White).  Three  of  the  eight  SE  development  models  were  used  to  assist  in 
determining  KSAs  required  across  different  SE  career  experience  levels.  The  SE 
development  models  used  are  as  follows:  INCOSE  UK,  MITRE,  and  the  SE  Workforce 
Development  NUWC.  Each  competency  model  used  had  various  career  levels  of 
experience  based  on  their  organizational  structure.  Bloom’s  Taxonomic  data  is  used  to 
map  these  KSAs  into  three  Naval  SE  experience  levels  and  domains  based  on  the 
following  experience  level  definitions: 

SE-01  Entry  Level  (0-3  years  of  work  experience) 

•  Able  to  understand  the  key  issues  and  their  implications.  They  are  able  to 
ask  relevant  and  constructive  questions  on  the  subject.  This  level  requires 
an  understanding  of  the  Systems  Engineering  role  within  the  enterprise. 

•  Example:  New  hires  enrolled  in  an  engineering  career  development 
program,  typically  able  to  complete  it  in  3  years. 

SE-02  Journey  Level  (3-10  years  of  work  experience) 

•  Displays  an  understanding  of  the  subject  but  may  require  minimal 
guidance  and  with  proper  training  and  opportunity  will  be  able  to  provide 
guidance  and  advice  to  others. 

•  Example:  GS-12  engineers  who  are  working  in  systems  engineering. 

SE-03  Expert  Level  (10-12+  years  of  work  experience) 

•  Contains  extensive  and  substantial  practical  experience  and  applied 
knowledge  of  the  subject. 

•  Example:  Senior  systems  engineers  who  are  leading  systems  engineering 
teams  and  possibly  act  as  a  chief  system  engineer. 

The  NPS  team  decided  to  address  the  mapping  of  these  competencies  across 
proficiency  levels  in  an  effort  to  create  a  foundation  for  SE  career  development  within  the 
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DON.  Each  of  the  KSAs  was  mapped  to  one  of  three  specific  career  levels  designated  as 
SE-01  Entry  Level,  SE-02  Journey  Level  or  SE-03  Expert  Level  as  shown  in  Figure  7. 


GS-l  to  8 


COMPOSE  Structure 


1 


Expert 

Level 


Figure  7.  COMPOSE  Career  Development  Levels 

This  was  accomplished  by  first  defining  each  of  the  KSAs  according  to  Bloom’s 
Taxonomy  of  Educational  Objectives,  which  is  discussed  in  more  detail  later  in  this 
section.  Specifically,  the  NPS  team  used  the  version  of  Bloom’s  Taxonomy  by  Krathwohl 
as  shown  in  Figure  8.  This  rating  and  mapping  further  strengthened  the  COMPOSE 
model  with  the  inclusion  of  yet  another  valid  information  source. 


23 


Figure  8.  Krathwohl’s  Bloom’s  Taxonomy 
(from  MMI  n.d.) 

Bloom’s  Taxonomy  for  the  Cognitive  Domain  provides  hierarchical  outcome 
categories  or  levels  that  range  from  simple  to  complex  thought  processes.  Once  the 
Bloom’s  level  for  each  of  the  KSAs  was  identified,  the  KSAs  in  each  competency  were 
divided  into  the  three  career  levels  by  assigning  the  KSAs  with  lower  Bloom’s  level 
ratings  to  the  SE-01  Entry  Level,  the  KSAs  with  intermediate  Bloom’s  level  ratings  to 
SE-02  Journey  Level  and  the  KSAs  with  higher  Bloom’s  level  ratings  to  the  SE-03 
Expert  Level.  Finally,  the  potential  sources  of  the  learning  are  partitioned  into  categories 
where  the  most  appropriate  learning  and  assessment  can  take  place.  Training  is  focused 
on  learning  that  is  applied  to  the  narrow  context  in  which  the  task  is  accomplished. 
Education  is  broader  and  focuses  learning  on  concepts  and  not  as  much  in  the  concrete 
accomplishment  of  specific  tasks.  The  tasks  in  education  are  “authentic”  at  best,  covering 
learning  that  can  be  transferred  into  many  different  contexts.  On  the  job  experience  is 
very  specific  to  the  accomplishment  of  a  task  within  a  specific  way  that  an  organization 
desires  it  to  be  completed  in  a  “real”  situation.  Professional  development  is  targeted  at 
learning  opportunities  that  are  education  and  training  based.  However,  professional 
development  has  a  narrow  scope  in  the  context  of  the  specific  development  within  a 
community  of  practice.  An  example  of  this  model  showing  Competency  11.0  Tools  and 
Techniques  is  illustrated  in  Appendix  B;  it  is  also  shown  in  Figure  9. 
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Knowledge  Skills  &  Abilities  (KSAs) 

Cognitive  &  Affective  Skill  Levels 

Education 

Training 

On  the  Job  Experience 

Remember,  Understand,  Receive  Phenomena  &  Respond  to  Phenomena 

Understand  the  scope  and  limitations  of  models  and  simulations,  including 
definition,  implementation  and  analysis 

X 

Understand  the  need  for  system  representations 

X 

Understand  the  different  types  of  modeling  and  simulation 

X 

Support  the  M&S  specialist  in  creating  and  validating  models 

X 

Support  systems  studies  and  analyses 

X 

Participate  in  networked  and  federated  M&S  developments  (e.g.,  training 
exercise  support  or  simulation  war  games),  with  assistance  from  the 
specialist 

X 

Know  which  models,  simulations,  or  decision  support  tools  can  be  used  to 
support  your  analysis,  evaluation,  instruction,  training  or  experiment 

X 

Know  various  models  &  simulations  and  when  it  is  beneficial  to  integrate 
them 

X 

Know  the  right  model  or  simulation  to  meet  your  specific  needs 

X 

Know  decision  support  tools,  models,  or  simulations  that  are  applicable  to 
your  job 

X 

Collaborate  with  the  specialist  to  run  M&S  scenarios,  based  on  current  and 
future  operational  capabilities 

X 

Assist  the  specialist  with  collecting  data  and  formulating  assumptions  to 
create  and  validate  simple  simulation  models  (e.g.,  operational  capabilities, 
networking,  computing  resources,  and  processes) 

X 

Apply,  Analyze,  &  Value 

Demonstrate  candidate  modeling  and/or  simulation  approach  (e.g., 
constructive,  virtual,  and  live  synthetic  environments)  while  working  with 
the  specialist 

X 

Build  prototypes  or  design  experiments  to  test  system  concepts  and  their 
feasibility 

X 

Evaluate,  Create,  Organize  &  Characterize  by  a  Value 

Survey  existing  data  and  previous  modeling  efforts  to  incorporate  previous 
M&S  capabilities  into  the  current  effort 

X 

Identify  potential  integration  and  interoperability  links  within  and  between 
modeling  and  simulation  tools  and  synthetic  environments 

X 

Figure  9.  The  COMPOSE  Model  Education,  Training  and  On  the  Job  Experience 
for  Competency  1 1.0  Tools  &  Techniques  Entry  Career  Level  Example 


The  third  iteration  of  the  COMPOSE  model  verification  was  initiated  by  research 
to  analyze  and  to  determine  to  what  extent  the  DAU  SPRDE-SE  certification  curriculum 
provides  the  basis  for  defining  KSAs  to  support  the  development  for  DOD  systems 
engineers.  As  an  initial  part  of  the  analysis,  654  Course  Learning/  Performance 
Objectives  (CL/PO)  and  Enabling  Learning  Objects  (ELO)  for  seven  DAU  SPRDE-SE 
Level  III  required  courses  were  identified.  These  CL/POs  and  ELOs  were  written  in  a 
format  similar  to  format  of  the  KSAs  in  the  COMPOSE  model.  The  NPS  Team  defined 
the  CL/POs  and  ELOs  as  KSAs  and  added  them  to  the  COMPOSE  model.  They  were 
also  mapped  to  competencies  in  the  COMPOSE  model  and  to  the  Graduate  Resource 
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Curriculum  for  Systems  Engineering  The  Graduate  Reference  Curriculum  for  SE 
(GRCSE)  Bloom’s  levels.  (Alexander  2013). 

The  COMPOSE  model  was  used  by  SPAWAR  Systems  Center  (SSC)  Atlantic  as 
a  foundation  to  develop  their  systems  engineering  competency  model.  Since  SPAWAR 
is  part  of  the  DON,  this  shows  that  the  COMPOSE  model  is  useful  as  intended  for 
tailoring  to  specific  implementations.  SPAWAR  being  able  to  use  the  COMPOSE  model 
is  verification  that  organizations  within  the  DOD  can  use  the  COMPOSE  model  by 
tailoring  to  meet  the  organization’s  goals.  In  the  SPAWAR  example,  the  KSAs  and 
categorization  will  help  determine  exactly  how  education  and  training  (structured  and 
unstructured)  can  best  be  utilized  to  maximize  the  effectiveness  of  systems  engineers 
(Walter  2013). 

The  SPRDE  Career  Development  model  was  changed  to  the  ENG  Career 
Development  model  in  September  2013  (Kendall  2013).  The  fourth  and  most  recent 
update  of  the  COMPOSE  model  was  initiated  when  the  2013  ENG  Competency  Update 
was  released.  This  refresh  added  KSAs  and  required  a  revision  of  the  KSA  nomenclature 
in  the  COMPOSE  model.  As  a  result  of  this  update  the  COMPOSE  model  has  a  total  of 
41  competencies  and  2,914  KSAs. 

B.  ALIGNING  INFORMATION  GATHERED 

Within  the  eight  original  competency  models  used  as  a  foundation  to  create  the 
COMPOSE  model,  competency  KSA  statements  were  not  defined  in  a  consistent  way. 
Some  models  had  overlap,  for  example,  “Able  to  guide  a  new  practitioner”  is  a  KSA  that 
was  originally  aligned  with  eight  competencies,  technical  planning,  acquisition, 
integration,  validation,  transition,  verification,  architecture  design  and  systems 
engineering  leadership.  The  NPS  approach  was  that  KSA  should  occur  in  only  one 
competency,  and  for  this  this  KSA  should  only  be  aligned  to  one  called  “Coaching  and 
Mentoring”  similar  to  the  SPRDE  competency  model  approach.  Some  models  had  more 
competencies  than  others.  All  of  the  competency  models  were  initially  re-categorized 
based  on  the  DAU  SE  Competency  model  of  forty-one  competencies  in  an  effort  to  tailor 
the  model  specifically  to  DOD.  Although  each  of  the  reference  competency  models  had 
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similar  purposes,  each  was  tailored  to  the  individual  organization.  This  tailoring  resulted 
in  several  different  formatting  styles  to  include  Microsoft  (MS)  Excel  tables,  Adobe 
reports  and  MS  PowerPoint  presentations.  This  tailoring  also  resulted  in  some  of  the 
models  using  electronic  format  while  others  were  in  the  paper  format.  These  differences 
in  format  and  formatting  styles  required  steps  to  harmonize  the  final  model. 

The  first  step  was  to  identify  the  format  that  would  best  fit  the  COMPOSE  model. 
When  determining  the  best  format,  it  was  important  to  think  about  who  would  be  using 
the  model  and  the  best  format  for  them.  It  was  also  equally  important  to  ensure  that  the 
format  of  the  model  could  accommodate  several  updates  and  changes.  It  was  determined 
to  initially  use  an  Excel  spreadsheet  model  that  could  be  provided  to  any  organization  to 
use  or  to  alter  the  model  to  easily  meet  their  organizational  needs.  This  spreadsheet  could 
also  handle  several  iterations  and  is  useful  for  making  updates  to  and  changing  the  model 
to  meet  specific  needs.  Based  on  input,  it  was  also  decided  to  use  a  report  format  that 
would  incorporate  tables  automatically  generated  in  the  Excel  spreadsheet.  These  tables 
could  easily  be  incorporated  into  a  more  formal  Competency  Report  output. 

The  spreadsheet  was  organized  in  columns.  The  column  headings  include  the 
COMPOSE  model  Competency,  KSAs,  Bloom’s  Cognitive,  Bloom’s  Affective, 
Proficiency  Level  and  Experience  Level.  Each  KSAs  was  entered  into  an  individual  row 
and  filled  out  appropriately.  The  spreadsheet  has  the  ability  to  filter  on  any  competency, 
KSAs  and  proficiency  level. 
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COMPOSE  Competen T 

COMPOSE  KSA  E 

COMPOSE  Category  £ 

COMPOSE  Experience  Lei 

Krathwohl  2002 Cogniti T 

Affective  ^ 

Cognitive  &  Affective  Domains 
(Krathwohl  2002)  £ 

Source 

21.0  ACQUISITION 

sponsor/customer  information  for  the 
development  of  acquisition  plans. 

Technical  &  Technical  Management 

SE-1  ENTRY  LEVEL 

Respond  (RS) 

Remember,  Understand,  Receive 
Phenomena  &  Respond  to 
Phenomena 

On  the  Job  Experience 

21.0  ACQUISITION 

the  impact  of  supportability  issues  on 
system  readiness/performance  and  other 
functional  areas.  E.g.  contracts,  finance, 
systems  engineering  and  acquisition  logistics 

Technical  &  Technical  Management 

SE-2  JOURNEY  LEVEL 

Analyze  (AN) 

Apply,  Analyze,  &  Value 

Training 

21.0  ACQUISITION 

trade-offs  during  implementation 
involving  design  constraints,  budget, 
schedule,  and  changing  priorities  across 
projects  and  programs. 

Technical  Technical  Management 

SE-2  JOURNEY  LEVEL 

Analyze  (AN) 

Apply,  Analyze,  &  Value 

Education 

Figure  10.  COMPOSE  Model  V0.78 
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C.  SCREENING 

Now  that  all  of  the  data  was  in  the  same  format,  the  next  step  was  to  eliminate 
duplication  and  items  that  do  not  fit.  Unique  identifiers  were  given  to  each  of  the  KSAs 
rows.  If  any  of  the  deleted  KSAs  need  to  be  used  in  the  future,  they  can  be  restored  by 
using  the  assigned  unique  identifier  to  trace  it  back  to  the  original  source  of  the  KSAs. 

The  spreadsheet  version  of  the  COMPOSE  model  that  is  available  to 
organizations  is  a  cleaned  up  version  with  only  the  fields  that  are  required  to  create  a 
competency  model.  Additionally,  NPS  has  the  original  “in-house”  COMPOSE  that  has 
other  fields  like  the  original  competency  model  source,  original  KSAs  and  experience 
level. 

D.  INCORPORATING  DAU  SPRDE  CL/POS  &  ELOS 

A  total  of  654  course  learning/  performance  objectives  (CL/POs)  and  enabling 
learning  objects  (ELOs)  for  seven  DAU  SPRDE-SE  Level  III  required  courses  were 
identified.  These  CL/POs  and  ELOs  were  then  defined  as  KSAs  and  added  to  the 
COMPOSE  model.  These  DAU/SPRDE  KSAs  were  also  mapped  to  competencies  in  the 
COMPOSE  model  and  to  the  GRCSE  Bloom’s  levels  (Alexander  2013). 

Incorporation  of  the  DAU  CL/POs  and  ELOs  into  the  NPS  SE  competency  model 
required  considerable  manipulation.  First,  the  new  KSAs  were  reworded  to  incorporate 
the  same  Bloom’s  verbs  that  are  used  in  the  rest  of  the  COMPOSE  model.  Six  duplicates 
were  discovered  and  discarded.  A  total  of  607  DAU  KSAs  were  ultimately  added  to 
version  0.5  of  the  COMPOSE  model.  Figure  1 1  shows  how  the  DAU  CL/POs  and  ELOs 
were  incorporated  into  the  model  (Alexander  2013). 
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35.0  Domain  Specialization 
34.0  Facilities  Engineering 
33.0  Contract  Management 
32.0  Financial  Managenemt 
31.0  Individual  and  Interpersonal.. 

30.0  Systems  Thinking 
29.0  Professional  Ethics 
28.0  StrategicThinking 
£  27.0  Problem  Solving 

c  26.0  Communication 

0) 

■£  25.0  System  of  Systems 

g  24.0  Systems  Engineering  Leadership 
3  23.0  Acquisition 

c  22.0  Software  Engineering 

21.0  Interface  Management 
20.0  Technical  Data  Management 
i5  19.0  Risk  Management 

E  18.0  Requirements  Management 

OJ 

"K  17.0  Configuration  Management 

^  16.0  Technical  Assessment 

15.0  Technical  Planning 
14.0  Decision  Analysis 
13.0  Reliability,  Availability,  and.. 
12.0  System  Assurance 
11.0  Transition 
10.0  Validation 
9.0  Verification 
8.0  Integration 
7.0  Implementation 
6.0  Architecture  Design 
5.0  Requirements  Analysis 
4.0  Stakeholder  Requirements  Definition 
3.0  Safety  Assurance 
2.0  Modelingand  Simulation 
1.0  Technical  Basis  for  Cost 
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Figure  11.  Number  of  DAU  SPRDE-SE  CL/POs  in  each  System  Engineering 

Competency  (from  Alexander  2013) 


The  most  recent  version  0.78  of  the  COMPOSE  model  incorporates  all  of  the 
changes.  Analysis  of  the  KSAs  shows  that  23  percent  of  the  KSAs  within  the  COMPOSE 
model  are  derived  directly  from  the  ENG  Competency  model  as  shown  in  Figure  12.  The 
spread  of  KSAs  across  career  levels  is  shown  in  Figure  13.  By  inspection,  the  number  of 
KSAs  in  the  COMPOSE  model  that  were  originally  from  the  ENG  Competency  model  in 
each  career  level  seem  to  be  consistent. 
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ENG/SPRDE  within  COMPOSE 


ENC/SPRDE  IGA’S 


23% 

(664) 


77% 

(2250) 


Total  KSA's  in  COMPOSE 


(2914) 


Figure  12.  ENG  KSAs  within  COMPOSE  V0.78 


Breakdown  of  ENG/SPRDE  across 
Experience  Levels  in  COMPOSE 


Total  KSA's  in  COMPOSE 
y  ENG/SPRDE  KSA's 


Figure  13.  Breakdown  of  ENG  KSAs  across  experience  levels  in  the  COMPOSE 

model  VO. 78 
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E.  MAPPING  TO  FIT  BLOOM’S  TAXONOMY 

Each  KSAs  was  mapped  according  to  Bloom’s  Taxonomy  of  Educational 
Objectives.  There  are  several  interpretations  of  Bloom’s;  however,  Krathwohl’s  version 
of  Bloom’s  Taxonomy,  which  is  shown  in  Chapter  I,  was  used  for  the  mapping.  As 
discussed  in  Chapter  II,  only  the  cognitive  and  affective  domains  were  used  for  analysis 
in  the  COMPOSE  model.  The  NPS  Team  assumed  that  the  psychomotor  is  not  relevant 
in  this  study  because  it  does  not  apply  to  engineering  in  any  useful  way. 

F.  IDENTIFYING  CAREER  LEVELS 

To  address  the  application  of  these  competencies  across  proficiency  levels,  each 
of  the  KSAs  was  mapped  to  one  of  three  specific  career  levels  designated  as  SE-1  Entry 
Level,  SE-2  Journey  Level  or  SE-3  Expert  Level.  Bloom’s  Taxonomy  provides 
hierarchical  outcome  categories  or  levels  that  range  from  simple  to  complex  thought 
processes.  Once  the  Bloom’s  level  for  each  KSAs  was  identified,  the  KSAs  in  each 
competency  were  divided  into  the  three  career  levels  by  assigning  the  KSAs  with  lower 
Bloom’s  level  ratings  to  the  SE-1  Entry  Level,  the  KSAs  with  intermediate  Bloom’s  level 
ratings  to  SE-2  Journey  Level  and  the  KSAs  with  higher  Bloom’s  level  ratings  to  the  SE- 
3  Expert  Level.  This  is  just  a  starting  point,  as  the  assignment  of  various  KSAs  to 
different  Bloom’s  levels  will  be  conducted  by  any  organization  in  their  implementation. 

G.  IDENTIFYING  POTENTIAL  COMPETENCY  LEARNING  SOURCES 

The  competencies  were  categorized  as  to  whether  they  would  be  developed  by 
“Education”,  “Training;”  and  “On  the  Job  Experience”.  An  example  of  this  model 
showing  Competency  11.0  Tools  and  Techniques  is  illustrated  in  Appendix  A.  This  step 
was  important  in  the  analysis  because  it  would  provide  an  initial  mapping  to  assist 
organizations  in  designing  SE  career  development  plans  and  provide  undergraduate  and 
graduate  SE  programs  a  baseline  that  identifies  the  KSAs  employers  expect  SE  to  have 
obtained  from  their  various  education  and  training  programs,  including  undergraduate 
and  graduate  education.  This  information  could  be  used  to  ensure  SE  educational 
programs  learning  objectives  meet  the  requirements  of  the  workforce  or  the  “customer”. 
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H.  COMPETENCY  SUMMARY 


The  COMPOSE  model  VO. 78  includes  all  of  the  competencies,  both  technical  and 
professional  were  left  in  the  model  in  an  effort  to  make  it  easier  for  an  organization  to 
tailor  the  model  to  meet  its  needs.  Future  naval  system  command  SME  ranking  of 
importance  in  a  follow  on  research  project  will  further  determine  what  KSAs  should  be  in 
the  final  model. 

I.  CHAPTER  SUMMARY 

This  chapter  presented  a  detailed  description  of  how  the  COMPOSE  model  was 
developed,  the  foundational  framework  used  and  how  the  original  information  gathered 
was  aligned.  The  information  from  the  eight  reference  competency  models  was 
combined  based  on  similarities,  un-useful  data  was  eliminated,  DAU  SPRDE  CL/POs 
and  ELOs  were  incorporated,  training  levels  were  identified  and  KSAs  were  mapped  to 
fit  Bloom’s  Taxonomy.  Currently,  the  COMPOSE  model  has  41  competencies,  2,914 
KSAs  and  three  notional  skill  levels.  The  next  chapter  will  analyze  the  findings. 
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IV.  DATA  AND  ANALYSIS 


This  chapter  discusses  some  trends  and  findings  identified  when  analyzing  both 
the  technical  and  professional  portions  of  the  COMPOSE  model.  This  chapter  will  also 
discuss  the  differences  and  similarities  between  the  cognitive  and  affective  levels. 

A.  PRIMARY  RESEARCH:  COMPOSE  V0.78  DATA 

When  analyzing  the  COMPOSE  model,  67%  of  the  KSAs  were  aligned  with  the 
cognitive  domain  while  33%  were  aligned  with  the  affective  domain  as  shown  in  Figure  14. 


COMPOSE  Cognitive  and  Affective 
Breakout 


■  Cognitive 

■  Affective 


Figure  14.  COMPOSE  Overall  Model’s  Cognitive  and  Affective  Breakout 


Within  the  cognitive  domain  about  23%  of  the  KSAs  are  aligned  with  Remember, 
11%  Understand  and  42%  Application  as  shown  in  figures  15-16. 
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Bloom's  Cognitive  Levels  within  the 

COMPOSE 

8% 

8%  23% 

■  Remember  (R  ) 

■  Understand  (U) 

8% 

■  Apply  (AP) 

11% 

■  Analyze  (AN) 

42% 

■  Evaluate  (EV) 

■  Create  (C ) 

Figure  15.  Bloom’s  Cognitive  Levels  within  the  COMPOSE  Overall  Model 


Bloom's  Cognitive  Levels  within  the 
COMPOSE 


Figure  16.  Bar  Chart  of  Bloom’s  Cognitive  Levels  within  the  COMPOSE  Overall 

Model 
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Within  the  affective  domain  about  71%  of  the  KSAs  are  aligned  with  Receiving 
and  Responding  and  17%  with  Valuing  as  shown  in  figures  13-14. 


Bloom's  Affective  Levels  within  the 

COMPOSE 

30/0  4%  50/0 

Receive  (RC) 

17% 

Respond  (RS) 

■  Value  (V) 

Organize  (OR) 

71% 

■  Characterize  (CH) 

Figure  17.  Bloom’s  Affective  Levels  within  the  COMPOSE  Overall  Model 


Bloom's  Affective  Levels  within  the 

COMPOSE  Overall  Model 

Characterization  (CH) 

42 

Organization  (OR) 

25 

Valuing  (V) 

175 

Responding  (RS) 

724 

Receiving  (RC) 

65 

( 

)  100  200  300  400  500  600  700  800 

Figure  18.  Bar  Chart  of  Bloom’s  Affective  Levels  within  the  COMPOSE  Overall 

Model 
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B.  ANALYSIS 


The  COMPOSE  model  is  primarily  comprised  of  KSAs  associated  with  the 
cognitive  domain.  Within  the  cognitive  domain  about  34%  of  the  KSAs  are  aligned  with 
Remembering  and  Understanding  and  42%  Applying.  This  implies  that  a  great  portion  of 
the  COMPOSE  model  relies  heavily  on  applying  the  prior  knowledge  learned.  Within 
the  affective  domain  about  71%  of  the  KSAs  are  aligned  with  Receiving  and  Responding 
and  17%  with  Valuing.  Although  the  COMPOSE  model  consists  of  mostly  KSAs  in  the 
cognitive  domain,  it  is  also  evident  that  to  be  a  competent  SE  it  is  important  to  have 
knowledge,  critical  thinking  and  the  development  of  intellectual  skills  from  the  cognitive 
domain  as  well  as  the  emotions,  feelings  and  attitudes  that  contribute  to  interpersonal 
skills  from  the  affective  domain. 

C.  SUMMARY 

Analysis  of  the  cognitive  and  affective  levels  using  Bloom’s  Taxonomy  shows 
that  the  majority  of  the  competency  model  is  aligned  within  the  cognitive  domain. 
Application  is  42%  of  the  KSAs  mapped  to  the  cognitive  domain  as  shown  in  Figure  15. 
In  the  affective  domain,  responding  is  71%  of  the  KSAs  as  shown  in  Figure  17. 
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V.  FINDINGS/RESULTS 


This  thesis  addressed  the  need  for  a  competency  model  as  a  solution  to  the  gap 
between  the  current  SE  competency  models  and  a  SE  position  description  for  the  DOD.  This 
chapter  will  present  the  findings  and  results  after  analyzing  the  COMPOSE  model. 

A.  PRIMARY  RESEARCH  FINDINGS 

Figures  19-21  shows  the  evolution  of  KSAs  across  career  levels  within  the 
cognitive  domain  for  the  COMPOSE  model.  Initially,  in  the  SE-01  Career  Level  for  the 
Technical  Competency  model,  the  majority  of  the  KSAs  are  Remember  and  Understand 
based  within  the  cognitive  domain,  which  makes  sense  since  these  are  lower  level 
cognitive  domains  that  can  be  gathered  by  training  and  education.  SE-01  is  composed  of 
43%  Remembering  and  18%  Understanding.  As  the  career  progresses  in  SE-02  and  SE- 
03  Career  Levels,  the  focus  shifts  to  Application.  At  this  stage  in  the  career  development, 
the  individual  is  required  to  apply  what  was  learned  to  do  his/her  job.  SE-02  is 
comprised  of  48%  Application  and  9%  Understanding,  while  SE-03  is  53%  Application 
and  7%  Analyzing.  Figure  22  illustrates  the  trend  of  the  Bloom’s  Cognitive  levels  within 
the  COMPOSE  model. 


Bloom's  Cognitive  Levels  within 


the  SE-01  Level  of  COMPOSE 


■  Understand  (U) 
Apply  (AP) 

■  Analyze  (AN) 

■  Evaluate  (EV) 

■  Create  (C ) 


Figure  19.  Bloom’s  Cognitive  Levels  within  the  COMPOSE  model  (SE-01) 
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Bloom's  Cognitive  Levels  within 

the  SE-02  Level  of  COMPOSE 
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■  Apply  (AP] 
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■  Evaluate  (EV] 
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Figure  20.  Bloom’s  Cognitive  Levels  within  the  COMPOSE  model  (SE-02) 


Bloom's  Cognitive  Levels  within 
the  SE-03  Level  of  COMPOSE 
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Figure  21.  Bloom’s  Cognitive  Levels  within  the  COMPOSE  model  (SE-03) 
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Figure  22.  Trend  of  Bloom’s  Cognitive  Levels  within  the  COMPOSE  model,  the  key  is  the  same  as  Figures  19-21. 


The  evolution  of  KSAs  across  career  levels  within  the  affective  domain  for  the 
COMPOSE  model  is  represented  in  figures  23-25.  It  seems  ideal  that  toward  the 
beginning  of  the  SE-01  and  SE-02  career  levels,  a  majority  of  the  KSAs  deal  with 
receiving  and  responding  within  the  affective  domain  because  these  are  lower  levels 
within  the  affective  domains  that  are  classified  as  part  of  individual’s  personality  traits. 
The  SE-01  Career  Level  is  composed  of  81%  Responding  and  9%  Receiving.  The  SE- 
02  Level  is  comprised  of  mostly  Responding  and  Valuing.  The  SE-03  is  representative 
of  Valuing,  Characterization  and  Responding.  Figure  26  illustrates  the  trend  of  Bloom’s 
Affective  levels  within  the  COMPOSE  model. 


Bloom's  Affective  Levels  within 

the  SE-01  Level  of  COMPOSE 

31b  1% 

«b  9% 

®  Rsccrva  {RCj 
■  Respond  (RS) 

■  VaX*  (v) 

■  Organise  (OR) 

am 

■  Guracunz*  (CH) 

Figure  23.  Bloom’s  Affective  Levels  within  the  COMPOSE  model  (SE-01) 
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Bloom's  Affective  Levels  within 

the  SE-02  Level  of  COMPOSE 

3%  2%2% 

Receive  (RC] 

17% 

' 

Respond  (RS] 

■  Value  (V) 

76% 

Organize  (OR] 

■  Characterize  (CH] 

Figure  24.  Bloom’s  Affective  Levels  within  the  COMPOSE  model  (SE-02) 


Bloom's  Affective  Levels  within 

the  SE-03  Level  of  COMPOSE 
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■  Characterize  (CH] 

Figure  25.  Bloom’s  Affective  Levels  within  the  COMPOSE  model  (SE-03) 
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SE-01  Level  of  COMPOSE 

SE-02  Level  of  COMPOSE 

SE-03  Level  of  COMPOSE 
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3%  2%  2% 

3% 
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31%  53% 

81% 
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Figure  26.  Trend  of  Bloom’s  Affective  Levels  within  the  COMPOSE  model,  the  key  is  the  same  as  figures  23-25. 


B.  OTHER  FINDINGS 


As  previously  stated,  the  COMPOSE  model  is  continuously  evolving.  SPAWAR 
has  taken  the  COMPOSE  VO. 5  model  as  a  foundation  to  create  their  SE  Competency 
model.  Figure  27  shows  that  about  14%  of  the  COMPOSE  was  used  directly.  The  model 
is  still  in  the  early  stages  of  development  and  although  one  is  unable  to  calculate  exactly, 
a  large  amount  of  the  remaining  86%  was  also  used,  but  it  was  tailored  to  meet  the  needs 
of  the  SPAWAR  specifically.  This  validates  that  the  COMPOSE  model  can  be  used  as  a 
foundation  and  tailored  to  meet  an  organization’s  needs  while  developing  a  SE 
Competency  model.  In  the  future  the  COMPOSE  model  will  be  shared  with  other 
organizations  and  the  capability  to  track  the  amount  of  the  COMPOSE  model  used 
directly  and  indirectly  in  a  newly  developed  SE  Competency  model  will  be  incorporated. 


NPS  SE  Model  can  be  used  as  a  foundation  and  tailored  to 
meet  organizations  needs  while  developing  a  SE 
Competency  Model:SPAWAR  Example 


COMPOSE  VO. 5 
Model  KSAs 
(2257) 


14% 


Figure  27.  SPAWAR  Example 


The  KSAs  taken  directly  from  the  COMPOSE  VO. 5  model  and  used  in  the 
SPAWAR  model  are  shown  in  Figure  28.  The  competency  model  has  evolved  since  then. 
This  analysis  is  based  on  the  COMPOSE  VO. 5  model,  the  model  has  evolved  and  the 
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most  recent  version  of  the  model  COMPOSE  VO. 78  model  has  updated  competencies. 
Additionally,  the  VO. 5  Model  is  aligned  to  GRCSE  Bloom’s  Taxonomy,  while  the  VO. 78 
Model  is  aligned  to  Krathwohl’s  Bloom’s  Update.  Figure  29  compares  the  KSAs  mapped 
to  the  cognitive  SWAWAR’s  model  with  the  COMPOSE  VO. 5  model  using  GRCSE 
Bloom’s  Taxonomy.  In  the  COMPOSE  VO. 5  model  competencies  25.  System  of 
Systems,  5.0  Requirements  Analysis,  16.0  Technical  Assessment  and  6.0  Architecture 
Design  Competencies  represent  more  than  10%  of  KSAs  from  the  COMPOSE  that 
SPAWAR  used  in  their  SE  Competency  model.  Within  the  overlap,  when  comparing  the 
cognitive  domains  in  both  models,  some  similarities  and  differences  are  evident  as  shown 
in  Figure  37.  Walter’s  work  also  reveals  that  the  majority  of  the  KSAs  in  SPAWAR’s  SE 
Competency  model  require  Application  as  shown  in  Figure  29  (Walter  2013). 
Additionally,  Knowledge  is  very  important  (38%)  in  the  SPAWAR  model,  while 
Comprehension  is  important  (14%)  in  the  COMPOSE  model. 


SE  Competency  Analysis:NPS  SE 
Competency  KSA's  SPAWAR  Used 


14.0  DECISION  ANALYSIS 
11.0  TRANSITION 
23.0  ACQUISITION 
22.0  SOFTWARE  ENGINEERING 
3.0  SAFETY  ASSURANCE 
13.0  RAM 
7.0  IMPLEMENTATION 
8.0  INTEGRATION 
25.0  SYSTEM  OF  SYSTEMS 
5.0  REQUIREMENTS  ANALYSIS 
16.0  TECHNICAL  ASSESSMENT 
6.0  ARCHITECTURE  DESIGN 
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Figure  28.  KSAs  from  the  COMPOSE  model  used  by  SPAWAR 


46 


GRCSE  Cognitive  Levels  within  the 
SPAWAR  Career  Development 
Model 


7%  4% 

■  Knowedge(K) 

2% 

■  Comprehension 

(CO) 

38% 

■  Aop  cat  on  ; ap| 

48% 

■  Anayas  (AN) 

■  Synthesis  (S) 

GRCSE  Cognitive  Levels  within 
the  COMPOSE  Model 


7% 


53% 


Figure  29.  SPAWAR  vs.  COMPOSE  model  Cognitive  Domain 

Alexander’s  work  emphasizes  that  in  general,  DAU  covers  the  knowledge  and 
comprehension  levels  of  Bloom’s  taxonomy  sufficiently.  The  majority  of  KSAs  are 
knowledge  and  comprehension  as  shown  in  Figure  30.  Therefore,  for  KSAs  that  are 
associated  with  basic  DOD-generic  (as  opposed  to  SSC  Atlantic-specific)  knowledge  and 
comprehension,  it  makes  sense  for  DAU  training  to  be  the  preferred  KSAs  development 
method  (Alexander  2013). 
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Figure  30.  Cognitive  Levels  of  DAU  SPRDE-SE  Level  II  Curriculum  and  NPS  SE 

Competency  Model  (from  Alexander  2013) 
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C.  RESEARCH  QUESTIONS  FINDINGS 

While  studying  the  attributes  required  to  create  a  useful  naval  systems 
engineering  competency  model,  a  set  of  additional  secondary  research  questions  surfaced. 
The  first  secondary  research  question  is  to  identify  what  competencies  are  required  for 
naval  systems  engineering.  The  COMPOSE  model  identifies  forty-one  competencies 
required  for  naval  systems  engineering.  Secondly,  it  pinpoints  what  KSAs  are  required 
for  development  of  naval  systems  engineering  competencies  at  various  levels.  The  model 
has  over  2,914  KSAs  mapped  across  the  forty-one  competencies.  Finally,  how  does  the 
use  of  Bloom’s  Taxonomy  for  KSAs  definition  relate  to  competency  development  along 
a  career  path?  After  analyzing  the  COMPOSE  model,  in  the  SE-01  Career  Level  the 
KSAs  were  associated  with  the  lower  level  cognitive  and  affective  domains.  Knowledge, 
comprehension,  receiving  and  responding  are  all  competencies  that  can  be  gathered  by 
training  and  education.  As  the  career  progresses  in  SE-02  and  SE-03  Career  Levels,  the 
focus  shifts  to  application.  At  this  stage  in  the  career  development,  the  individual  is 
required  to  apply  what  was  learned  to  do  his/her  job. 

In  the  affective  domain  it  seems  ideal  that  toward  the  beginning  of  the  SE-01  and 
SE-02  career  levels,  a  majority  of  the  KSAs  deal  with  receiving  and  responding  within 
the  affective  domain.  These  are  lower  level  affective  domains  that  are  classified  as  a  part 
of  the  individual’s  personality  traits.  The  SE-01  Career  Level  is  composed  of  81% 
Responding  and  9%  Receiving.  The  SE-02  level  is  comprised  of  mostly  Responding  and 
Valuing.  The  SE-03  level  is  representative  of  Valuing,  Characterization  and  Responding. 

The  primary  research  question  is  to  explore  what  a  systems  engineering 
competency  model  would  consist  of  at  the  general  naval  systems  commands  to  use  for 
career  planning  and  tracking  competency  development.  The  COMPOSE  model  MS  Excel 
spreadsheet  is  an  interactive  model  composed  of  KSAs  aligned  to  specific  competencies 
across  Bloom’s  cognitive  and  affective  domains.  To  address  the  application  of  these 
competencies  across  proficiency  levels,  each  KSAs  was  mapped  to  one  of  three  specific 
career  levels  designated  as  SE-1  Entry  Level,  SE-2  Journey  Level  or  SE-3  Expert  Level. 
The  source  to  obtain  competence  for  each  KSAs  was  also  categorized  as  to  whether  they 
would  be  developed  by  “Education  and  Training;”  “On  the  Job  Experience;”  or 
“Professional  Development.” 
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Initially,  in  the  SE-01  Career  Level  for  the  Technical  Competency  model,  the 
majority  of  the  KSAs  are  Knowledge  and  Comprehension  based  within  the  cognitive 
domain,  which  makes  sense  since  these  are  lower  level  cognitive  domains  that  can  be 
gathered  by  training  and  education.  SE-01  is  composed  of  43%  Remembering  and  18% 
Understanding.  As  the  career  progresses  in  SE-02  and  SE-03  Career  Levels,  the  focus 
shifts  to  Application.  At  this  stage  in  the  career  development,  the  individual  is  required  to 
apply  what  was  learned  to  do  his/her  job. 

The  evolution  of  KSAs  across  career  levels  within  the  affective  domain  for  the 
COMPOSE  professional  model.  Interestingly,  when  taking  a  look  at  the  affective  domain 
with  the  Professional  Skills  Competency  model,  it  is  clear  that  the  three  career  levels  all 
look  very  similar.  Majority  of  the  KSAs  are  categorized  as  Responding  and  Valuing. 
Throughout  each  of  the  Career  Levels  Responding  and  Valuing  are  the  main  focuses. 

Other  findings  include  that  the  KSAs  taken  directly  from  the  COMPOSE  VO. 5 
and  used  in  the  SPAWAR  model.  SPAWAR  used  more  than  10%  of  KSAs  from  the 
COMPOSE  0.5  model  in  their  SE  Competency  Model.  There  is  a  14%  overlap  between 
the  COMPOSE  and  SPAWAR  competency  models.  Within  that  overlap,  when  comparing 
the  cognitive  domains  in  the  model  the  majority  of  the  KSAs  require  Application. 
Additionally,  Knowledge  is  very  important  (38%)  in  the  SPAWAR  model,  while 
Comprehension  is  important  in  the  COMPOSE  model. 

D.  CHAPTER  SUMMARY 

This  chapter  discussed  trends  and  findings  when  analyzing  the  COMPOSE  model 
to  include  how  the  COMPOSE  model  was  used  as  a  foundation  for  SPAWAR’ s  SE 
Competency  model.  Similarities  between  SPAWAR’ s  model  and  the  COMPOSE  model 
were  provided  using  pie  charts  and  bar  graphs. 
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VI.  SUMMARY,  CONCLUSIONS,  RECOMMENDATIONS  AND 

FURTHER  RESEARCH 


This  chapter  presents  a  brief  summary  of  the  research  conducted  in  order  to 
address  the  need  for  a  SE  position  description  and  career  development  plan.  The 
development  of  the  COMPOSE  model  is  summarized,  conclusions  from  the  analysis  of 
the  data  are  presented,  recommendations  about  the  next  steps  to  take  are  provided  and 
further  areas  of  research  are  discussed. 

A.  SUMMARY 

The  COMPOSE  model  developed  by  the  NPS  systems  engineering  (SE)  team 
includes  SE  career  development  to  include:  knowledge,  skills,  abilities  and  behaviors, 
and  related  education,  training,  and  experience  needed.  It  also  provides  a  way  to  define 
career  paths  for  systems  engineers  (jobs,  assignments,  and  timing)  based  on  Bloom’s 
Taxonomy  Cognitive  and  Affective  domains.  The  COMPOSE  model  and  Bloom’s 
Taxonomy  both  provide  hierarchical  outcome  categories  or  levels  that  range  from  simple 
to  complex  thought  processes.  Once  the  Bloom’s  level  for  each  KSAs  was  identified,  the 
KSAs  in  each  competency  were  divided  into  the  three  career  levels  by  assigning  the 
KSAs  with  lower  Bloom’s  level  ratings  to  the  SE-1  Entry  Level,  the  KSAs  with 
intermediate  Bloom’s  level  ratings  to  SE-2  Journey  Level  and  the  KSAs  with  higher 
Bloom’s  level  ratings  to  the  SE-2  Expert  Level. 

Because  of  the  way  that  the  COMPOSE  model  is  formatted,  the  COMPOSE 
model  can  be  used  by  organizations  to  identify  KSAs  pertinent  to  the  development  of 
systems  engineers.  The  model  will  also  allow  organizations  to  formulate  competency 
development  plans  for  the  professional  development  of  systems  engineers.  Finally,  the 
model  will  contribute  to  the  guidance  for  development  of  graduate  and  undergraduate 
curricula  in  systems  engineering. 

The  COMPOSE  model  encompasses  eight  different  systems  engineering 
competency  models,  which  includes  INCOSE  UK,  Boeing,  NASA,  DAU  SPRDE  and 
NUWC  Newport.  They  combine  and  harmonize  with  Krathwohl’s  version  of  Bloom’s 
Taxonomy  based  on  affinity  in  the  KSAs.  The  KSAs  are  re-aligned  to  fit  the  model  by 
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eliminating  duplication,  items  that  do  not  fit  and  are  re-categorized  based  on  how  each 
KSAs  is  written.  The  COMPOSE  model  has  a  total  of  forty-one  competencies  and  2,914 
KSAs. 

Initially,  in  the  SE-01  Career  Level  for  the  Technical  Competency  model,  the 
majority  of  the  KSAs  are  Remember  and  Understand  based  within  the  cognitive  domain, 
which  makes  sense  since  these  are  lower  level  cognitive  domains  that  can  be  gathered  by 
training  and  education.  SE-01  is  composed  of  43%  Remembering  and  18% 
Understanding.  Analysis  results  indicate  that  when  it  comes  to  technical  competency 
within  Systems  Engineering,  at  entry-level  positions  lower  level  KSAs  from  the  cognitive 
domain  are  required.  As  the  career  level  increases,  so  does  the  complexity  of  the  KSAs 
within  the  cognitive  domain.  The  SE-02  and  SE-03  career  level’s  focus  shifts  to 
Application.  At  this  stage  in  the  career  development,  the  individual  is  required  to  apply 
what  was  learned  to  do  his/her  job.  SE-02  is  comprised  of  48%  Application  and  9% 
Understanding,  while  SE-03  is  53%  Application  and  7%  Analyzing.  Although  the  model 
is  still  being  refined,  the  initial  results  indicate  that  it  can  be  used  as  a  foundation  for  an 
organization  to  tailor  to  develop  a  systems  engineering  competency  development  model. 

B.  CONCLUSIONS  AND  RECOMMENDATIONS 

The  Competency  model  for  the  Profession  of  Systems  Engineering  (COMPOSE) 
consists  of  41  competencies  and  over  2,914  KSAs  defined  using  Bloom’s  Taxonomy 
Cognitive  and  Affective  domains  that  span  across  three  experience  levels.  Proficiency 
levels  within  each  experience  level  are  also  identified.  The  model  is  formatted  in  a  MS 
Excel  spreadsheet  which  could  be  provided  to  any  organization  to  use  and  tailor  the 
model  to  meet  their  organizational  needs  easily.  This  spreadsheet  was  designed  to  be 
highly  interactive,  especially  when  using  the  extensive  filtering  capabilities  of  the 
spreadsheet.  Organizations  should  be  able  to  take  the  model  and  use  it  as  a  foundation  to 
create  a  competency  model  that  meets  their  organizational  needs.  Based  on  the  input, 
tables  are  automatically  generated  in  the  excel  spreadsheet.  These  tables  could  easily  be 
incorporated  into  a  more  formal  competency  report  document  format. 

SPAWAR  SC  Atlantic  has  taken  the  COMPOSE  VO. 5  model  as  a  foundation  to 
create  their  SE  Competency  model.  About  14%  of  the  COMPOSE  VO. 5  model  was  used 
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directly.  The  model  is  still  in  the  early  stages  of  development  and  although  we  are  unable 
to  calculate  exactly,  a  large  amount  of  the  remaining  86%  was  also  used,  but  it  was 
tailored  to  meet  the  needs  of  the  SPAWAR  specifically.  This  validates  that  the 
COMPOSE  model  can  be  used  as  a  foundation  and  tailored  to  meet  an  organizations 
needs  while  developing  a  SE  Competency  model.  In  the  future  it  is  recommended  to 
share  the  COMPOSE  model  with  other  organizations  to  identify  which  attributes  are  used 
directly  and  indirectly  in  order  to  develop  a  new  SE  Competency  model. 

C.  FURTHER  RESEARCH 

While  studying  the  attributes  required  to  create  a  useful  naval  systems 
engineering  competency  model  opportunities  for  further  research  efforts  were  identified. 

•  Future  Research  Opportunity  1 :  How  do  these  competencies  and  KS As 
trend  along  a  career  path  for  various  systems  engineers? 

•  Future  Research  Opportunity  2:  Evaluate  the  model  using  Office  of 
Personnel  (OPM)  guidelines. 

•  Future  Research  Opportunity  3:  How  could  a  systems  engineering 
competency  model  be  used  to  inform  undergraduate  and  graduate 
education  programs  for  systems  engineering? 

•  Future  Research  Opportunity  4:  How  could  an  assessment  procedure 
within  the  competency  model  be  beneficial  in  addition  including  a 
framework  for  validating  assessments? 

•  Future  Research  Opportunity  5:  It  would  be  interesting  to  analyze 
exactly  how  the  COMPOSE  was  used  in  a  given  organization  after 
adopted.  For  example,  what  percentage  of  the  model  was  used  to  create 
job  descriptions,  initiate  training  requirements  or  create  performance 
evaluation  measures? 

•  Future  Research  Opportunity  6:  Re-categorize  the  model  to  compliment 
the  DAU  SE  Competency  model. 
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APPENDIX  A.  EXAMPLES  OF  CAREER  LEVELS  FOR  THE  V0.78 

COMPOSE  MODEL 


Entry  Level:  1 1.0  Tools  and  Techniques 


Knowledge  Skills  &  Abilities  (KSAs) 

Cognitive  &  Affective  Skill  Levels 

Education 

Training 

On  the  Job  Experience 

Remember,  Understand,  Receive  Phenomena  &  Respond  to  Phenomena 

Understand  the  scope  and  limitations  of  models  and  simulations,  including 
definition,  implementation  and  analysis 

X 

Understand  the  need  for  system  representations 

X 

Understand  the  different  types  of  modeling  and  simulation 

X 

Support  the  M&S  specialist  in  creating  and  validating  models 

X 

Support  systems  studies  and  analyses 

X 

Participate  in  networked  and  federated  M&S  developments  (e.g.,  training 
exercise  support  or  simulation  war  games),  with  assistance  from  the 
specialist 

X 

Know  which  models,  simulations,  or  decision  support  tools  can  be  used  to 
support  your  analysis,  evaluation,  instruction,  training  or  experiment 

X 

Know  various  models  &  simulations  and  when  it  is  beneficial  to  integrate 
them 

X 

Know  the  right  model  or  simulation  to  meet  your  specific  needs 

X 

Know  decision  support  tools,  models,  or  simulations  that  are  applicable  to 
your  job 

X 

Collaborate  with  the  specialist  to  run  M&S  scenarios,  based  on  current  and 
future  operational  capabilities 

X 

Assist  the  specialist  with  collecting  data  and  formulating  assumptions  to 
create  and  validate  simple  simulation  models  (e.g.,  operational  capabilities, 
networking,  computing  resources,  and  processes) 

X 

Apply,  Analyze,  &  Value 

Demonstrate  candidate  modeling  and/or  simulation  approach  (e.g., 
constructive,  virtual,  and  live  synthetic  environments)  while  working  with 
the  specialist 

X 

Build  prototypes  or  design  experiments  to  test  system  concepts  and  their 
feasibility 

X 

Evaluate,  Create,  Organize  &  Characterize  by  a  Value 

Survey  existing  data  and  previous  modeling  efforts  to  incorporate  previous 
M&S  capabilities  into  the  current  effort 

X 

Identify  potential  integration  and  interoperability  links  within  and  between 
modeling  and  simulation  tools  and  synthetic  environments 

X 
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Journey  Level:  1 1.0  Tools  and  Techniques 


Knowledge  Skills  &  Abilities  (KSAs) 

On  the  Job  Experience  1 

Cognitive  &  Affective  Skill  Levels 

Education 

Training 

Remember,  Understand,  Receive  Phenomena  &  Respond  to  Phenomena 

Understand  the  risks  of  using  models  and  simulations  which  are  outside  the 
validated  limits 

X 

Models  systems  of  varying  complexities  in  their  environments 

X 

Know  how  to  run  and  interpret  the  output  of  modeling  and  simulation  tools,  to 
provide  insight  or  training  to  real  world  situations 

X 

Know  how  to  initialize  the  modeling  and  simulation  tools 

X 

Collaborate  with  the  specialist  to  interpret  the  results  of  various  M&S  scenarios, 
based  on  current  and  future  operational  capabilities 

X 

Select  appropriate  tools  and  techniques  for  functional  analysis 

X 

Define  a  process  that  includes  requirements  for  appropriate  tools  and  techniques 
for  architectural  design 

X 

Use  scenarios  to  determine  robustness 

X 

Contribute  to  definition  of  design  and  product  constraints  for  a  subsystem  or 
simple  project 

X 

Apply,  Analyze,  &  Value 

Apply  models,  simulations  and/or  decision  support  tools 

X 

Evaluate  models,  simulations  and/or  decision  support  tools 

X 

Evolve  models  and/or  simulations 

X 

Employ  models  and/or  simulations 

X 

Integrate  models  and/or  simulations 

X 

Manage  models  and/or  simulations 

X 

Apply  models  and/or  simulations 

X 

Develop  models  and/or  simulations 

X 

Determine  requirements  for  the  application  of  models  and/or  simulations 

X 

Define  an  appropriate  representation  of  a  system  or  system  element 

X 

Collaborate  with  the  specialist  to  develop  assumptions  and  scenarios,  and  create 
and  validate  complex  simulation  models  (e.g.,  operational  capabilities, 
networking,  computing  resources,  and  processes) 

X 

Collaborate  with  the  M&S  specialist  to  identify  Approach,  create  and  validate 
models,  interpret  results,  and  participate  in  cooperative  modeling  arrangements 

X 

Apply  real-world  data  in  models  or  simulations  for  computer  generated  forces, 
mathematical  modeling,  physical  modeling,  scientific  research,  and  statistical 
analysis 

X 

Analyze  models,  simulations  and/or  decision  support  tools 

X 

Perform  Value  Engineering,  an  organized,  systematic  technique  to  analyze  the 
functions  of  systems,  equipment,  facilities,  services,  and  supplies  to  ensure  they 
achieve  their  essential  functions  at  the  lowest  life-cycle  cost  consistent  with 
required  performance,  reliability,  quality,  and  safety 

X 

Perform  sustainability  analyses  to  reduce  system  total  ownership  cost  by 
uncovering  previously  hidden  or  ignored  life-cycle  costs,  leading  to  more 
informed  decisions  earlier  in  the  acquisition  life  cycle 

X 

Evolve  the  authoritative  model  of  the  system  under  development 

X 

Employ  the  authoritative  model  of  the  system  under  development 

X 

Integrate  the  authoritative  model  of  the  system  under  development 

X 

Manage  the  authorities  model  of  the  system  under  development 

X 

Develop  the  authorities  model  of  the  system  under  development 

X 

Apply  the  authoritative  model  of  the  system  under  development 

X 

Interpret  modeling  or  simulation  results  to  more  fully  explore  concepts,  refine 
system  characteristics/designs,  assess  overall  system  performance,  and  better 
inform  acquisition  program  decisions 

X 

Provide  technical  basis  for  program  budgets  that  reflect  program  phase 
requirements  and  best  practices  using  knowledge  of  Earned  Value  Management, 
cost  drivers,  risk  factors,  and  historical  documentation  (e.g.  hardware, 
operational  software,  lab/support  software) 

X 

Provide  technical  basis  for  comprehensive  cost  Estimate  that  reflect  program 
phase  requirements  and  best  practices  using  knowledge  of  Earned  Value 
Management,  cost  drivers,  risk  factors,  and  historical  documentation  (e.g. 
hardware,  operational  software,  lab/support  software) 

X 
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Knowledge  Skills  &  Abilities  (KSAs) 

Cognitive  &  Affective  Skill  Levels 

Education 

Training 

On  the  Job  Experience 

Evaluate,  Create,  Organize  &  Characterize  by  a  Value 

Suggest  collaboration  with  other  organizations  to  establish  integration  and 
interoperability  within  and  between  modeling  and  simulation  tools  and  synthetic 
environments 

X 

Plan  models  or  simulations  to  drive  exercises 

X 

Identify  Approach,  tools,  and  techniques  to  describe,  analyze,  and  synthesize 
complicated  and  complex  systems 

X 

Execute  models  or  simulations  to  drive  exercises 

X 

Compare  the  strengths  and  limitations  of  modeling  and  simulation  Approach, 
identify  Approach  that  fit  the  scope,  and  define  the  visualization  approach  while 
working  with  the  specialist 

X 

Ensure  credibility  of  models  or  simulations  by  adhering  to  and  applying  sound 
verification,  validation  and  accreditation  (VV&A)  practices 

X 

Create  a  comprehensive,  integrated  model  that  Describe  systems  of  varying 
complexities  in  their  environments,  including  systems  dynamics  (e.g.  human, 
organizational  and  technical  dynamics) 

X 

Develop  innovative  solutions  that  include  S&T  developmental  prototyping, 
experimentation  and  visualization  techniques 

X 
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Expert  Level:  1 1.0  Tools  and  Techniques 


Knowledge  Skills  &  Abilities  (KSAs) 

Cognitive  &  Affective  Skill  Levels 

Education 

Training 

On  the  Job  Experience 

Remember,  Understand,  Receive  Phenomena  &  Respond  to  Phenomena 

Present  to  the  sponsor/customer  and  key  stakeholders  a  comprehensive, 
integrated  model  that  Describe  systems  of  varying  complexities  in  their 
environments,  including  systems  dynamics  (e.g.  human,  organizational  and 
technical  dynamics) 

X 

Describe  the  VV&A  process 

X 

Describe  the  M&S  planning  process  as  a  support  tool  for  systems  engineering 

X 

Describe  the  M&S  hierarchy 

X 

Define  the  strategy  and  approach  to  be  adopted  for  the  modeling  and 
simulation  of  a  system  or  system  element 

X 

Apply,  Analyze,  &  Value 

Recommend  the  scope  of  modeling,  simulation  and  analysis  activities  within 
and  across  projects/programs,  with  input  from  the  modeling  and  simulation 
specialist 

X 

Recommend  modeling  and  simulation  Approach  within  and  across  projects, 
programs,  and  enterprises,  including  the  visualization  approach 

X 

Recommend  M&S  scope,  Approach,  and  changes  to  operational  capabilities, 
and  Facilitate  cooperative  modeling  arrangements 

X 

Recommend  changes  to  current  and  future  operational  capabilities  based  on 
modeling  and  simulation  results 

X 

Provide  expert  technical  advice  on  the  verification,  validation  and  accreditation 
of  models  or  simulations 

X 

Provide  expert  technical  advice  on  model  or  simulation  architectures 

X 

Manage/supervise  the  development  and  application  of  models  and/or 
simulations  for  a  Program 

X 

Lead  networked  and  federated  M&S  developments  (e.g.  major  training 
exercises  or  simulation  war  games),  with  assistance  from  the  specialist 

X 

Guide  the  formulation  of  assumptions  and  scenarios  developed  by  the 
specialist  to  create  complex  simulation  models  (e.g.,  operational  capabilities, 
networking,  computing  resources,  and  processes) 

X 

Explain  the  difference  between  fidelity  and  resolution 

X 

Explain  the  difference  between  a  model  and  a  simulation 

X 

Explain  the  application  of  modeling  and  simulation  to  systems  engineering. 

X 

Update  modeling  and  simulation  standards,  policy,  and  guidance  for  an 
organization 

X 

Review  modeling  and  simulation  standards,  policy,  and  guidance 

X 

Develop  modeling  and  simulation  standards,  policy,  and  guidance 

X 

Demonstrate  a  full  understanding  of  complex  simulations  for  a  system  or 
system  element 

X 

Apply  doctrinal  and  operational  knowledge  during  simulation  exercise 
execution 

X 

Advise  on  the  suitability  and  limitations  of  models  and  simulations 

X 
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APPENDIX  B.  SE  COMPETENCY  OBJECTIVES 


SE  Competency:  Cognitive  &  Affective  Components 


Faculty 

Cliff  Whitcomb 

Sponsor 

DASN 

Partners 

Naval  SYSCOMS 

Student 

Corina  White 

Grad 

June  2014 

Objectives 

•  Develop  competency  model  and  identify  and  evaluate  the 
cognitive  and  affective  domains  associated  with  SE  KSA. 

•  Define  KSAs  using  Bloom's  Taxonomy:  Cognitive  and  Affective 
Domains. 

•  Harmonize  SE  Competency  Career  Development  Model  with 
GRCSE. 

•  Develop  an  approach  and  methodology  to  obtain  baseline 
information  needed  for  the  Naval  SE  Competency  Model  life  cycle 
development  (knowledge,  skills,  abilities  and  behaviors,  and 
related  education,  training,  and  experience  needed). 

•  Define  career  paths  for  systems  engineers  (jobs,  assignments,  and 
timing). 

•  Document  results. 


Approach 

•  Develop  Excel  spreadsheet  based  SE 
Competency  Career  Development  Model 

•  Utilize  Bloom's  Taxonomy  to  define  KSA  in 
cognitive  and  affective  domains 

•  Analyze  the  cognitive/affective  skills 
needed  to  develop  as  a  proficient  SE 

•  Document  results  in  a  thesis 
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APPENDIX  C.  KRATHWOHL  COGNITIVE  AND  AFFECTIVE 


DOMAINS 

Cognitive  Domain  (adapted  from  Bloom’s  fjmmm 

LEVEL 

EXAMPLE  COMPETENCIES 

Remember  (C) 

•  The  student  is  able  to  recite  the  definitions  of  "system"  and  "emergence"  and  state  the  connection  between  them. 

•  The  student  is  able  to  describe  the  notion  of  product  system  architecture  and  state  the  impact  architecture  may  have  on  system  development  success. 

Understand  (U) 

•  The  student  is  able  to  explain,  in  a  very  general  way,  the  conditions  under  which  a  system  development  team  might  choose  to  use  a  waterfall  (or 
iterative,  incremental,  or  spiral)  life  cycle  model. 

•  The  student  is  able  to  explain  the  range  of  cases  for  which  a  particular  systems  modeling  approach  is  applicable. 

Apply  (AP) 

•  Given  the  operational  concept  and  requirements  of  a  simple  system  along  with  a  specified  budget  and  required  completion  time,  the  student  is  able  to 
choose  (and  to  provide  a  rudimentary  justification  for  the  choice)  a  particular  life  cycle  model  to  address  the  project;  e.g.,  waterfall,  iterative,  incremental, 
or  spiral. 

•  The  student  is  able  to  construct  a  simple  model  of  a  defined  system  that  would  demonstrate  understanding  of  the  relationship  of  the  primary  factors 
included  in  the  model. 

Analyze(AN) 

•Given  a  simple  requirements  document  and  a  domain  model  for  an  application,  the  student  is  able  to  critique  the  domain  model. 

•Given  the  operational  concept  of  a  system  along  with  a  requirements  document,  a  budget,  a  schedule,  a  choice  of  a  development  process,  and  a 
justification  of  the  use  of  that  process  for  the  project,  the  student  is  able  to  find  and  explain  errors  in  the  justification  and/or  in  the  choice  of  the  process. 
•The  student  can  analyze  the  effectiveness  of  a  simple  model  of  a  system  to  describe  the  behavior  of  that  system  and  identify  errors  or  weaknesses  in  the 
model  arising  from  the  assumptions  about  the  system  embedded  in  the  model. 

Evaluate  (EV) 

•  Given  a  detailed  requirements  document  and  a  well-constructed  domain  model  for  a  system,  the  student  is  able  to  design  at  least  one  basic  architecture 
for  the  system. 

•  Given  an  operational  concept,  requirements,  architecture,  and  detailed  design  documents  for  a  system,  the  student  is  able  to  construct  a  complete 
implementation  plan  and  provide  a  cogent  argument  that  if  the  implementation  of  the  architecture  or  detailed  design  is  performed  according  to  the  plan, 
then  the  result  will  be  a  system  that  satisfies  the  requirements,  fulfills  the  operational  concept,  and  will  be  completed  within  the  budget  and  schedule. 

•  The  student  can  develop  and  use  a  model  of  a  simple  system  where  the  system  is  described  by  architecture  to  determine  the  capability  of  the  system 
represented  by  the  model  and  to  explore  desirable  parameters  of  model  elements. 

Create  (C) 

•  Given  an  operational  concept,  requirements,  architecture,  a  detailed  design,  and  an  implementation  plan,  including  budget  and 
schedule,  for  a  system,  as  well  as  a  feasibility  argument  for  the  implementation  plan,  the  student  is  able  to  assess  the  plan  and  to  either 
explain  why  the  feasibility  argument  is  valid  or  why  and  where  it  is  flawed  with  regard  to  any  of  the  claims  regarding  implementation  of 
the  requirements,  fulfillment  of  the  operational  concept,  or  the  ability  to  be  completed  within  budget  and  schedule. 

•  Given  a  simple  system,  the  student  is  able  to  plan  a  test  and  evaluation  method  to  perform  a  verification  and  validation  process  of  that 
system  against  the  requirements  of  the  system  and  the  need  description  associated  with  the  system. 

•  Given  a  simple  system  and  a  test  and  evaluation  plan  of  the  system,  the  student  is  able  to  determine  that  the  results  that  would  be 
produced  through  use  of  the  test  and  evaluation  plan  will  yield  a  useful  verification  and  validation  of  the  system. 

1/17/14 


25 


61 


Affective  Domain  (adapted  from  Bloom’s) 


LEVEL 

EXAMPLE  COMPETENCIES  M 

Receive  (RC) 

•  The  student  accepts  that  customer  or  user  perception  of  the  quality  of  a  system  is  the  fundamental  determinant  of  W 
system  quality. 

•  The  student  accepts  that  customers  do  not  always  fully  describe  what  they  want  or  need,  and  that  there  is  a  difference 
between  what  customers  say  they  want  and  what  they  actually  need. 

•  The  student  is  able  to  describe  the  value  of  the  SE  approach  to  design. 

Respond  (RS) 

•  The  student  learns  how  to  ask  questions  to  elicit  the  unstated  desires  of  a  stakeholder  who  is  seeking  a  system 
development. 

•  The  student  is  willing  to  try  the  SE  approach  on  a  small  project. 

Value  (V) 

•  The  student  believes  it  is  important  to  provide  system  solutions  that  satisfy  the  range  of  stakeholder  concerns  in  a 
manner  that  the  stakeholders  judge  to  be  good. 

•  The  student  believes  it  is  important  to  elicit  a  nuanced  description  of  what  stakeholders  desire  of  a  system  in  order  to 
provide  rich  knowledge  that  can  be  used  in  the  system  solution  development. 

•  The  student  believes  in  the  value  of  the  application  of  SE  principles  in  a  project,  even  in  the  face  of  advocates  for  other 
methods. 

•  The  student  recognizes  the  value  of  advancing  in  proficiency  in  SE  competencies. 

Organize  (OR) 

•  The  student  is  able  to  organize  a  coherent  framework  of  beliefs  and  understandings  to  support  use  of  a  SE  method  in  a 
project. 

•  The  student  has  a  coherent  framework  for  how  to  discuss  system  development  with  stakeholders  and  to  incorporate 
the  views  of  a  variety  of  stakeholders  in  a  balanced  manner. 

.Characterize  CH) 

•  The  student  will  routinely  approach  system  development  projects  with  a  SE  framework. 

•  The  student  will  routinely  evaluate  the  appropriate  tailoring  of  SE  processes  to  appropriately  address  the  specific 
characteristics  of  each  project. 

•  The  student  will  appropriately  weigh  the  views  of  all  stakeholders  and  seek  to  overcome  conflicts  between 
stakeholders  using  methods  that  are  technically  and  socially  appropriate. 
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APPENDIX  D.  KENDALL’S  MEMO 


THE  UNDER  SECRETARY  OF  DEFENSE 
30 1  O  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-3010 

SEP  1  6  2013 


MEMORANDUM  FOR:  SEE  DISTRIBUTION 

SUBJECT:  Sunsetting  of  the  Systems  Planning,  Research,  Development,  and  Engineering  - 

Program  Systems  Engineer  Acquisition  Workforce  Career  Path  and  the  Renaming  of 
the  Systems  Planning,  Research,  Development,  and  Engineering  Acquisition 
Workforce  Career  Field 

The  Deputy  Assistant  Secretary  of  Defense  for  Systems  Engineering  (DASD(SE)), 

Mr.  Stephen  Welby,  as  the  Functional  Leader  for  the  Systems  Planning,  Research,  Development, 
and  Engineering  (SPRDE)  career  field,  has  recommended  the  sunsetting  of  the  Program  Systems 
Engineer  (PSE)  acquisition  workforce  career  path.  Based  on  this  recommendation,  I  authorize 
the  sunsetting  of  the  SPRDE-PSE  career  path,  effective  October  1,  2013. 

Accordingly,  the  engineers  in  the  “SPRDE-PSE”  acquisition  workforce  career  path,  as 
well  as  all  personnel  coded  within  the  “SPRDE-Systems  Engineering”  acquisition  workforce 
career  path,  will  be  transitioned  to  a  consolidated  and  renamed  acquisition  workforce  career  field 
with  the  general  title  of  “Engineering.”  Renaming  this  career  field  will  emphasize  the 
contribution  of  these  professionals  to  Department  of  Defense  (DoD)  acquisition.  This  change  is 
effective  immediately. 

Those  acquisition  workforce  members  currently  coded  under  the  Defense  Civilian 
Personnel  Data  System  “W”  code  at  Level  III  in  the  PSE  career  path  may  keep  this  identifier. 

The  “W”  code  is  not  to  be  used  for  any  new  coding  after  PSE  is  sunsetted.  All  others  will  be 
recoded  to  the  “S”  code.  Component  Directors  for  Acquisition  Career  Management  will  recode 
the  PSE  acquisition  workforce/positions  as  quickly  as  possible.  The  Position  Category 
Description  for  PSE  will  be  eliminated  in  its  entirety. 

Members  of  the  SPRDE-PSE  acquisition  workforce  have  served  DoD  and  their 
respective  organizations  with  distinction,  and  this  change  in  no  way  reflects  on  the  value  they 
have  provided  to  their  organizations.  This  change  has  been  implemented  to  simplify  processes, 
reduce  cost,  and  leverage  the  SPRDE-PSE  workforce  expertise  across  the  larger  Engineering 
acquisition  workforce. 

Please  direct  questions  to  Mr.  Nicholas  Torelli,  Director,  ODASD(SE)/Mission 
Assurance,  at  703-695-2300  or  nicholas.m.torelli.civ@mail.mil. 


Frank  Kendall 


ACQUISITION, 
TECHNOLOGY 
AND  LOGISTICS 
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